{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "rNdWfPXCjTjY"
   },
   "source": [
    "# Introduction to Feature Columns \n",
    "**Learning Objectives**\n",
    "\n",
    "\n",
    "1. Load a CSV file using [Pandas](https://pandas.pydata.org/)\n",
    "2. Create an input pipeline using tf.data\n",
    "3. Create multiple types of feature columns\n",
    "\n",
    " \n",
    "\n",
    "## Introduction \n",
    "\n",
    "In this notebook, you classify structured data (e.g. tabular data in a CSV file) using [feature columns](https://www.tensorflow.org/tutorials/structured_data/feature_columns).  Feature columns serve as a bridge to map from columns in a CSV file to features used to train a model. In a subsequent lab, we will use [Keras](https://www.tensorflow.org/guide/keras) to define the model.\n",
    "\n",
    "Each learning objective will correspond to a __#TODO__ in the [student lab notebook](../labs/feat.cols_tf.data.ipynb) -- try to complete that notebook first before reviewing this solution notebook. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "K1y4OHpGgss7"
   },
   "source": [
    "## The Dataset\n",
    "\n",
    "We will use a small [dataset](https://archive.ics.uci.edu/ml/datasets/heart+Disease) provided by the Cleveland Clinic Foundation for Heart Disease. There are several hundred rows in the CSV. Each row describes   a patient, and each column describes an attribute. We will use this information to predict whether a patient has heart disease, which in this dataset is a binary classification task.\n",
    "\n",
    "Below is a [description](https://archive.ics.uci.edu/ml/machine-learning-databases/heart-disease/heart-disease.names) of this dataset. Notice there are both numeric and categorical columns.\n",
    "\n",
    ">Column| Description| Feature Type | Data Type\n",
    ">------------|--------------------|----------------------|-----------------\n",
    ">Age | Age in years | Numerical | integer\n",
    ">Sex | (1 = male; 0 = female) | Categorical | integer\n",
    ">CP | Chest pain type (0, 1, 2, 3, 4) | Categorical | integer\n",
    ">Trestbpd | Resting blood pressure (in mm Hg on admission to the hospital) | Numerical | integer\n",
    ">Chol | Serum cholestoral in mg/dl | Numerical | integer\n",
    ">FBS | (fasting blood sugar > 120 mg/dl) (1 = true; 0 = false) | Categorical | integer\n",
    ">RestECG | Resting electrocardiographic results (0, 1, 2) | Categorical | integer\n",
    ">Thalach | Maximum heart rate achieved | Numerical | integer\n",
    ">Exang | Exercise induced angina (1 = yes; 0 = no) | Categorical | integer\n",
    ">Oldpeak | ST depression induced by exercise relative to rest | Numerical | float\n",
    ">Slope | The slope of the peak exercise ST segment | Numerical | integer\n",
    ">CA | Number of major vessels (0-3) colored by flourosopy | Numerical | integer\n",
    ">Thal | 3 = normal; 6 = fixed defect; 7 = reversable defect | Categorical | string\n",
    ">Target | Diagnosis of heart disease (1 = true; 0 = false) | Classification | integer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "VxyBFc_kKazA"
   },
   "source": [
    "## Import TensorFlow and other libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "9dEreb4QKizj"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "TensorFlow version:  2.6.0\n"
     ]
    }
   ],
   "source": [
    "# You can use any Python source file as a module by executing an import statement in some other Python source file.\n",
    "# The import statement combines two operations; it searches for the named module, then it binds the results of that search\n",
    "# to a name in the local scope.\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "# Import matplotlib to visualize the model\n",
    "import matplotlib.pyplot as plt\n",
    "# Seaborn is a Python data visualization library based on matplotlib\n",
    "import seaborn as sns\n",
    "# %matplotlib inline sets the backend of matplotlib to the `inline` backend\n",
    "%matplotlib inline\n",
    "\n",
    "import tensorflow as tf\n",
    "\n",
    "\n",
    "from tensorflow import feature_column\n",
    "from tensorflow.keras import layers\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "print(\"TensorFlow version: \",tf.version.VERSION)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "KCEhSZcULZ9n"
   },
   "source": [
    "## Lab Task 1: Use Pandas to create a dataframe\n",
    "\n",
    "[Pandas](https://pandas.pydata.org/) is a Python library with many helpful utilities for loading and working with structured data. We will use Pandas to download the dataset from a URL, and load it into a dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "REZ57BXCLdfG"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>sex</th>\n",
       "      <th>cp</th>\n",
       "      <th>trestbps</th>\n",
       "      <th>chol</th>\n",
       "      <th>fbs</th>\n",
       "      <th>restecg</th>\n",
       "      <th>thalach</th>\n",
       "      <th>exang</th>\n",
       "      <th>oldpeak</th>\n",
       "      <th>slope</th>\n",
       "      <th>ca</th>\n",
       "      <th>thal</th>\n",
       "      <th>target</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>63</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>145</td>\n",
       "      <td>233</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>150</td>\n",
       "      <td>0</td>\n",
       "      <td>2.3</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>fixed</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>67</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>160</td>\n",
       "      <td>286</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>108</td>\n",
       "      <td>1</td>\n",
       "      <td>1.5</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>normal</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>67</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>120</td>\n",
       "      <td>229</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>129</td>\n",
       "      <td>1</td>\n",
       "      <td>2.6</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>reversible</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>37</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>130</td>\n",
       "      <td>250</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>187</td>\n",
       "      <td>0</td>\n",
       "      <td>3.5</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>normal</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>41</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>130</td>\n",
       "      <td>204</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>172</td>\n",
       "      <td>0</td>\n",
       "      <td>1.4</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>normal</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   age  sex  cp  trestbps  chol  fbs  restecg  thalach  exang  oldpeak  slope  \\\n",
       "0   63    1   1       145   233    1        2      150      0      2.3      3   \n",
       "1   67    1   4       160   286    0        2      108      1      1.5      2   \n",
       "2   67    1   4       120   229    0        2      129      1      2.6      2   \n",
       "3   37    1   3       130   250    0        0      187      0      3.5      3   \n",
       "4   41    0   2       130   204    0        2      172      0      1.4      1   \n",
       "\n",
       "   ca        thal  target  \n",
       "0   0       fixed       0  \n",
       "1   3      normal       1  \n",
       "2   2  reversible       0  \n",
       "3   0      normal       0  \n",
       "4   0      normal       0  "
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "URL = 'https://storage.googleapis.com/download.tensorflow.org/data/heart.csv'\n",
    "# Read a comma-separated values (csv) file into a DataFrame using the read_csv() function\n",
    "dataframe = pd.read_csv(URL)\n",
    "# Get the first five rows using the head() method\n",
    "dataframe.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 303 entries, 0 to 302\n",
      "Data columns (total 14 columns):\n",
      "age         303 non-null int64\n",
      "sex         303 non-null int64\n",
      "cp          303 non-null int64\n",
      "trestbps    303 non-null int64\n",
      "chol        303 non-null int64\n",
      "fbs         303 non-null int64\n",
      "restecg     303 non-null int64\n",
      "thalach     303 non-null int64\n",
      "exang       303 non-null int64\n",
      "oldpeak     303 non-null float64\n",
      "slope       303 non-null int64\n",
      "ca          303 non-null int64\n",
      "thal        303 non-null object\n",
      "target      303 non-null int64\n",
      "dtypes: float64(1), int64(12), object(1)\n",
      "memory usage: 33.3+ KB\n"
     ]
    }
   ],
   "source": [
    "# Get a concise summary of a DataFrame\n",
    "dataframe.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Split the dataframe into train, validation, and test\n",
    "\n",
    "The dataset we downloaded was a single CSV file. As a best practice, we will split this into train, validation, and test sets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "193 train examples\n",
      "49 validation examples\n",
      "61 test examples\n"
     ]
    }
   ],
   "source": [
    "# TODO 1a\n",
    "# Create test, validation and train samples from one dataframe with pandas.\n",
    "train, test = train_test_split(dataframe, test_size=0.2)\n",
    "train, val = train_test_split(train, test_size=0.2)\n",
    "print(len(train), 'train examples')\n",
    "print(len(val), 'validation examples')\n",
    "print(len(test), 'test examples')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Lab Task 2: Create an input pipeline using tf.data\n",
    "\n",
    "Next, we will wrap the dataframes with [tf.data](https://www.tensorflow.org/datasets). This will enable us  to use feature columns as a bridge to map from the columns in the Pandas dataframe to features used to train a model. If we were working with a very large CSV file (so large that it does not fit into memory), we would use tf.data to read it from disk directly. That is not covered in this lab."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [],
   "source": [
    "# A utility method to create a tf.data dataset from a Pandas Dataframe\n",
    "def df_to_dataset(dataframe, shuffle=True, batch_size=32):\n",
    "  dataframe = dataframe.copy()\n",
    "  labels = dataframe.pop('target')\n",
    "  ds = tf.data.Dataset.from_tensor_slices((dict(dataframe), labels)) # TODO 2a\n",
    "  if shuffle:\n",
    "    ds = ds.shuffle(buffer_size=len(dataframe))\n",
    "  ds = ds.batch(batch_size)\n",
    "  return ds"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [],
   "source": [
    "# A small batch size is used for demonstration purposes\n",
    "batch_size = 5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO 2b\n",
    "train_ds = df_to_dataset(train, batch_size=batch_size)\n",
    "val_ds = df_to_dataset(val, shuffle=False, batch_size=batch_size)\n",
    "test_ds = df_to_dataset(test, shuffle=False, batch_size=batch_size)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Understand the input pipeline\n",
    "\n",
    "Now that we have created the input pipeline, let's call it to see the format of the data it returns. We have used a small batch size to keep the output readable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Every feature: ['ca', 'thal', 'trestbps', 'restecg', 'oldpeak', 'exang', 'sex', 'age', 'slope', 'chol', 'fbs', 'thalach', 'cp']\n",
      "A batch of ages: tf.Tensor([49 68 41 51 63], shape=(5,), dtype=int32)\n",
      "A batch of targets: tf.Tensor([0 0 0 0 0], shape=(5,), dtype=int32)\n"
     ]
    }
   ],
   "source": [
    "# If you don't use take(1), all elements will eventually be fetched\n",
    "for feature_batch, label_batch in train_ds.take(1):\n",
    "  print('Every feature:', list(feature_batch.keys()))\n",
    "  print('A batch of ages:', feature_batch['age'])\n",
    "  print('A batch of targets:', label_batch)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "ttIvgLRaNoOQ"
   },
   "source": [
    "## Lab Task 3: Demonstrate several types of feature column\n",
    "TensorFlow provides many types of feature columns. In this section, we will create several types of feature columns, and demonstrate how they transform a column from the dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We will use this batch to demonstrate several types of feature columns\n",
    "example_batch = next(iter(train_ds))[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [],
   "source": [
    "# A utility method to create a feature column\n",
    "# and to transform a batch of data\n",
    "def demo(feature_column):\n",
    "  feature_layer = layers.DenseFeatures(feature_column)\n",
    "  print(feature_layer(example_batch).numpy())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Q7OEKe82N-Qb"
   },
   "source": [
    "### Numeric columns\n",
    "The output of a feature column becomes the input to the model. A [numeric column](https://www.tensorflow.org/api_docs/python/tf/feature_column/numeric_column) is the simplest type of column. It is used to represent real valued features. When using this column, your model will receive the column value from the dataframe unchanged."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NumericColumn(key='age', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None)\n"
     ]
    }
   ],
   "source": [
    "# Create a numeric feature column out of `age`\n",
    "age = feature_column.numeric_column(\"age\")\n",
    "tf.feature_column.numeric_column\n",
    "print(age)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "7a6ddSyzOKpq"
   },
   "source": [
    "### Let's have a look at the output:\n",
    "\n",
    "#### key='age'\n",
    "A unique string identifying the input feature. It is used as the column name and the dictionary key for feature parsing configs, feature Tensor objects, and feature columns.\n",
    "\n",
    "#### shape=(1,)\n",
    "In the heart disease dataset, most columns from the dataframe are numeric.  Recall that tensors have a rank.  \"Age\" is a \"vector\" or \"rank-1\" tensor, which is like a list of values. A vector has 1-axis, thus the shape will always look like this:  shape=(3,), where 3 is a scalar (or single number) and with 1-axis.  \n",
    "\n",
    "#### default_value=None\n",
    "A single value compatible with dtype or an iterable of values compatible with dtype which the column takes on during tf.Example parsing if data is missing. A default value of None will cause tf.io.parse_example to fail if an example does not contain this column. If a single value is provided, the same value will be applied as the default value for every item. If an iterable of values is provided, the shape of the default_value should be equal to the given shape.\n",
    "\n",
    "#### dtype=tf.float32\n",
    "defines the type of values. Default value is tf.float32. Must be a non-quantized, real integer or floating point type.\n",
    "\n",
    "\n",
    "#### normalizer_fn=None\n",
    "If not None, a function that can be used to normalize the value of the tensor after default_value is applied for parsing. Normalizer function takes the input Tensor as its argument, and returns the output Tensor. (e.g. lambda x: (x - 3.0) / 4.2). Please note that even though the most common use case of this function is normalization, it can be used for any kind of Tensorflow transformations.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:Layer dense_features_22 is casting an input tensor from dtype float64 to the layer's dtype of float32, which is new behavior in TensorFlow 2.  The layer has dtype float32 because it's dtype defaults to floatx.\n",
      "\n",
      "If you intended to run this layer in float32, you can safely ignore this warning. If in doubt, this warning is likely only an issue if you are porting a TensorFlow 1.X model to TensorFlow 2.\n",
      "\n",
      "To change all layers to have dtype float64 by default, call `tf.keras.backend.set_floatx('float64')`. To change just this layer, pass dtype='float64' to the layer constructor. If you are the author of this layer, you can disable autocasting by passing autocast=False to the base Layer constructor.\n",
      "\n",
      "[[60.]\n",
      " [58.]\n",
      " [55.]\n",
      " [54.]\n",
      " [51.]]\n"
     ]
    }
   ],
   "source": [
    "# Demo of a numeric feature column out of `age`\n",
    "demo(age)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "IcSxUoYgOlA1"
   },
   "source": [
    "### Bucketized columns\n",
    "Often, you don't want to feed a number directly into the model, but instead split its value into different categories based on numerical ranges. Consider raw data that represents a person's age. Instead of representing age as a numeric column, we could split the age into several buckets using a [bucketized column](https://www.tensorflow.org/api_docs/python/tf/feature_column/bucketized_column). Notice the one-hot values below describe which age range each row matches."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "wJ4Wt3SAOpTQ"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:Layer dense_features_23 is casting an input tensor from dtype float64 to the layer's dtype of float32, which is new behavior in TensorFlow 2.  The layer has dtype float32 because it's dtype defaults to floatx.\n",
      "\n",
      "If you intended to run this layer in float32, you can safely ignore this warning. If in doubt, this warning is likely only an issue if you are porting a TensorFlow 1.X model to TensorFlow 2.\n",
      "\n",
      "To change all layers to have dtype float64 by default, call `tf.keras.backend.set_floatx('float64')`. To change just this layer, pass dtype='float64' to the layer constructor. If you are the author of this layer, you can disable autocasting by passing autocast=False to the base Layer constructor.\n",
      "\n",
      "[[0. 0. 0. 0. 0. 0. 0. 0. 0. 1. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 0. 1. 0. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 0. 1. 0. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 1. 0. 0. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 1. 0. 0. 0.]]\n"
     ]
    }
   ],
   "source": [
    "# Create a bucketized feature column out of `age` with the following boundaries and demo it.\n",
    "age_buckets = tf.feature_column.bucketized_column(age, boundaries=[18, 25, 30, 35, 40, 45, 50, 55, 60, 65])\n",
    "demo(age_buckets) # TODO 3a\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "r1tArzewPb-b"
   },
   "source": [
    "### Categorical columns\n",
    "In this dataset, thal is represented as a string (e.g. 'fixed', 'normal', or 'reversible'). We cannot feed strings directly to a model. Instead, we must first map them to numeric values. The categorical vocabulary columns provide a way to represent strings as a one-hot vector (much like you have seen above with age buckets). The vocabulary can be passed as a list using [categorical_column_with_vocabulary_list](https://www.tensorflow.org/api_docs/python/tf/feature_column/categorical_column_with_vocabulary_list), or loaded from a file using [categorical_column_with_vocabulary_file](https://www.tensorflow.org/api_docs/python/tf/feature_column/categorical_column_with_vocabulary_file)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "DJ6QnSHkPtOC"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:Layer dense_features_24 is casting an input tensor from dtype float64 to the layer's dtype of float32, which is new behavior in TensorFlow 2.  The layer has dtype float32 because it's dtype defaults to floatx.\n",
      "\n",
      "If you intended to run this layer in float32, you can safely ignore this warning. If in doubt, this warning is likely only an issue if you are porting a TensorFlow 1.X model to TensorFlow 2.\n",
      "\n",
      "To change all layers to have dtype float64 by default, call `tf.keras.backend.set_floatx('float64')`. To change just this layer, pass dtype='float64' to the layer constructor. If you are the author of this layer, you can disable autocasting by passing autocast=False to the base Layer constructor.\n",
      "\n",
      "[[0. 0. 1.]\n",
      " [0. 1. 0.]\n",
      " [0. 0. 1.]\n",
      " [0. 1. 0.]\n",
      " [0. 1. 0.]]\n"
     ]
    }
   ],
   "source": [
    "# Create a categorical vocabulary column out of the\n",
    "# above mentioned categories with the key specified as `thal`.\n",
    "thal = tf.feature_column.categorical_column_with_vocabulary_list(\n",
    "      'thal', ['fixed', 'normal', 'reversible'])\n",
    "\n",
    "# Create an indicator column out of the created categorical column.\n",
    "thal_one_hot = tf.feature_column.indicator_column(thal)\n",
    "demo(thal_one_hot)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "dxQloQ9jOoXL"
   },
   "source": [
    "In a more complex dataset, many columns would be categorical (e.g. strings). Feature columns are most valuable when working with categorical data. Although there is only one categorical column in this dataset, we will use it to demonstrate several important types of feature columns that you could use when working with other datasets."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "LEFPjUr6QmwS"
   },
   "source": [
    "### Embedding columns\n",
    "Suppose instead of having just a few possible strings, we have thousands (or more) values per category. For a number of reasons, as the number of categories grow large, it becomes infeasible to train a neural network using one-hot encodings. We can use an embedding column to overcome this limitation. Instead of representing the data as a one-hot vector of many dimensions, an [embedding column](https://www.tensorflow.org/api_docs/python/tf/feature_column/embedding_column) represents that data as a lower-dimensional, dense vector in which each cell can contain any number, not just 0 or 1. The size of the embedding (8, in the example below) is a parameter that must be tuned.\n",
    "\n",
    "Key point: using an embedding column is best when a categorical column has many possible values. We are using one here for demonstration purposes, so you have a complete example you can modify for a different dataset in the future."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "hSlohmr2Q_UU"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:Layer dense_features_25 is casting an input tensor from dtype float64 to the layer's dtype of float32, which is new behavior in TensorFlow 2.  The layer has dtype float32 because it's dtype defaults to floatx.\n",
      "\n",
      "If you intended to run this layer in float32, you can safely ignore this warning. If in doubt, this warning is likely only an issue if you are porting a TensorFlow 1.X model to TensorFlow 2.\n",
      "\n",
      "To change all layers to have dtype float64 by default, call `tf.keras.backend.set_floatx('float64')`. To change just this layer, pass dtype='float64' to the layer constructor. If you are the author of this layer, you can disable autocasting by passing autocast=False to the base Layer constructor.\n",
      "\n",
      "[[ 0.26216975 -0.66194284  0.33328214 -0.09756625  0.20408471  0.57926923\n",
      "  -0.07685163  0.4386801 ]\n",
      " [-0.24602154  0.0877578   0.07975551  0.34634778  0.2708743  -0.6707659\n",
      "  -0.15825593 -0.08179379]\n",
      " [ 0.26216975 -0.66194284  0.33328214 -0.09756625  0.20408471  0.57926923\n",
      "  -0.07685163  0.4386801 ]\n",
      " [-0.24602154  0.0877578   0.07975551  0.34634778  0.2708743  -0.6707659\n",
      "  -0.15825593 -0.08179379]\n",
      " [-0.24602154  0.0877578   0.07975551  0.34634778  0.2708743  -0.6707659\n",
      "  -0.15825593 -0.08179379]]\n"
     ]
    }
   ],
   "source": [
    "# Notice the input to the embedding column is the categorical column\n",
    "# we previously created\n",
    "# Set the size of the embedding to 8, by using the dimension parameter\n",
    "thal_embedding = tf.feature_column.embedding_column(thal, dimension=8)\n",
    "demo(thal_embedding)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "urFCAvTVRMpB"
   },
   "source": [
    "### Hashed feature columns\n",
    "\n",
    "Another way to represent a categorical column with a large number of values is to use a [categorical_column_with_hash_bucket](https://www.tensorflow.org/api_docs/python/tf/feature_column/categorical_column_with_hash_bucket). This feature column calculates a hash value of the input, then selects one of the `hash_bucket_size` buckets to encode a string. When using this column, you do not need to provide the vocabulary, and you can choose to make the number of hash_buckets significantly smaller than the number of actual categories to save space.\n",
    "\n",
    "Key point: An important downside of this technique is that there may be collisions in which different strings are mapped to the same bucket. In practice, this can work well for some datasets regardless."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "YHU_Aj2nRRDC"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:Layer dense_features_26 is casting an input tensor from dtype float64 to the layer's dtype of float32, which is new behavior in TensorFlow 2.  The layer has dtype float32 because it's dtype defaults to floatx.\n",
      "\n",
      "If you intended to run this layer in float32, you can safely ignore this warning. If in doubt, this warning is likely only an issue if you are porting a TensorFlow 1.X model to TensorFlow 2.\n",
      "\n",
      "To change all layers to have dtype float64 by default, call `tf.keras.backend.set_floatx('float64')`. To change just this layer, pass dtype='float64' to the layer constructor. If you are the author of this layer, you can disable autocasting by passing autocast=False to the base Layer constructor.\n",
      "\n",
      "[[0. 0. 0. ... 0. 0. 0.]\n",
      " [0. 0. 0. ... 0. 0. 0.]\n",
      " [0. 0. 0. ... 0. 0. 0.]\n",
      " [0. 0. 0. ... 0. 0. 0.]\n",
      " [0. 0. 0. ... 0. 0. 0.]]\n"
     ]
    }
   ],
   "source": [
    "# Create a hashed feature column with `thal` as the key and 1000 hash buckets.\n",
    "thal_hashed = tf.feature_column.categorical_column_with_hash_bucket(\n",
    "      'thal', hash_bucket_size=1000)\n",
    "demo(tf.feature_column.indicator_column(thal_hashed))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "fB94M27DRXtZ"
   },
   "source": [
    "### Crossed feature columns\n",
    "Combining features into a single feature, better known as [feature crosses](https://developers.google.com/machine-learning/glossary/#feature_cross), enables a model to learn separate weights for each combination of features. Here, we will create a new feature that is the cross of age and thal. Note that `crossed_column` does not build the full table of all possible combinations (which could be very large). Instead, it is backed by a `hashed_column`, so you can choose how large the table is."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "oaPVERd9Rep6"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:Layer dense_features_27 is casting an input tensor from dtype float64 to the layer's dtype of float32, which is new behavior in TensorFlow 2.  The layer has dtype float32 because it's dtype defaults to floatx.\n",
      "\n",
      "If you intended to run this layer in float32, you can safely ignore this warning. If in doubt, this warning is likely only an issue if you are porting a TensorFlow 1.X model to TensorFlow 2.\n",
      "\n",
      "To change all layers to have dtype float64 by default, call `tf.keras.backend.set_floatx('float64')`. To change just this layer, pass dtype='float64' to the layer constructor. If you are the author of this layer, you can disable autocasting by passing autocast=False to the base Layer constructor.\n",
      "\n",
      "[[0. 0. 0. ... 0. 0. 0.]\n",
      " [0. 0. 0. ... 0. 0. 0.]\n",
      " [0. 0. 0. ... 0. 0. 0.]\n",
      " [0. 0. 0. ... 0. 0. 0.]\n",
      " [0. 0. 0. ... 0. 0. 0.]]\n"
     ]
    }
   ],
   "source": [
    "# Create a crossed column using the bucketized column (age_buckets)\n",
    "# the categorical vocabulary column (thal), and 1000 hash buckets.\n",
    "crossed_feature = tf.feature_column.crossed_column([age_buckets, thal], hash_bucket_size=1000)\n",
    "demo(tf.feature_column.indicator_column(crossed_feature))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "ypkI9zx6Rj1q"
   },
   "source": [
    "## Choose which columns to use\n",
    "We have seen how to use several types of feature columns. Now we will use them to train a model. The goal of this tutorial is to show you the complete code (e.g. mechanics) needed to work with feature columns. We have selected a few columns to train our model below arbitrarily.\n",
    "\n",
    "Key point: If your aim is to build an accurate model, try a larger dataset of your own, and think carefully about which features are the most meaningful to include, and how they should be represented."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "4PlLY7fORuzA"
   },
   "outputs": [],
   "source": [
    "feature_columns = []\n",
    "\n",
    "# numeric cols\n",
    "# Create a feature column out of the header using a numeric column.\n",
    "for header in ['age', 'trestbps', 'chol', 'thalach', 'oldpeak', 'slope', 'ca']:\n",
    "  feature_columns.append(feature_column.numeric_column(header))\n",
    "\n",
    "# bucketized cols\n",
    "# Create a bucketized feature column out of the age column using the following boundaries.\n",
    "age_buckets = feature_column.bucketized_column(age, boundaries=[18, 25, 30, 35, 40, 45, 50, 55, 60, 65])\n",
    "feature_columns.append(age_buckets)\n",
    "\n",
    "# indicator cols\n",
    "# Create a categorical vocabulary column out of the below categories with the key specified as `thal`.\n",
    "thal = feature_column.categorical_column_with_vocabulary_list(\n",
    "      'thal', ['fixed', 'normal', 'reversible'])\n",
    "thal_one_hot = feature_column.indicator_column(thal)\n",
    "feature_columns.append(thal_one_hot)\n",
    "\n",
    "# embedding cols\n",
    "# Create an embedding column out of the categorical vocabulary\n",
    "thal_embedding = feature_column.embedding_column(thal, dimension=8)\n",
    "feature_columns.append(thal_embedding)\n",
    "\n",
    "# crossed cols\n",
    "# Create a crossed column using the bucketized column (age_buckets),\n",
    "# the categorical vocabulary column (thal), and 1000 hash buckets.\n",
    "crossed_feature = feature_column.crossed_column([age_buckets, thal], hash_bucket_size=1000)\n",
    "crossed_feature = feature_column.indicator_column(crossed_feature)\n",
    "feature_columns.append(crossed_feature)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "M-nDp8krS_ts"
   },
   "source": [
    "### How to Input Feature Columns to a Keras Model\n",
    "Now that we have defined our feature columns, we now use a [DenseFeatures](https://www.tensorflow.org/versions/r2.0/api_docs/python/tf/keras/layers/DenseFeatures) layer to input them to a Keras model.  Don't worry if you have not used Keras before.  There is a more detailed video and lab introducing the Keras Sequential and Functional models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "6o-El1R2TGQP"
   },
   "outputs": [],
   "source": [
    "# Create a Keras DenseFeatures layer and pass the feature_columns\n",
    "feature_layer = tf.keras.layers.DenseFeatures(feature_columns)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "8cf6vKfgTH0U"
   },
   "source": [
    "Earlier, we used a small batch size to demonstrate how feature columns worked. We create a new input pipeline with a larger batch size."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "gcemszoGSse_"
   },
   "outputs": [],
   "source": [
    "batch_size = 32\n",
    "train_ds = df_to_dataset(train, batch_size=batch_size)\n",
    "val_ds = df_to_dataset(val, shuffle=False, batch_size=batch_size)\n",
    "test_ds = df_to_dataset(test, shuffle=False, batch_size=batch_size)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "bBx4Xu0eTXWq"
   },
   "source": [
    "## Create, compile, and train the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "_YJPPb3xTPeZ"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train for 7 steps, validate for 2 steps\n",
      "Epoch 1/5\n",
      "7/7 [==============================] - 1s 157ms/step - loss: 1.1446 - accuracy: 0.6580 - val_loss: 1.4723 - val_accuracy: 0.4694\n",
      "Epoch 2/5\n",
      "7/7 [==============================] - 0s 10ms/step - loss: 0.7330 - accuracy: 0.6632 - val_loss: 0.5254 - val_accuracy: 0.7143\n",
      "Epoch 3/5\n",
      "7/7 [==============================] - 0s 10ms/step - loss: 0.4610 - accuracy: 0.7565 - val_loss: 0.4916 - val_accuracy: 0.7755\n",
      "Epoch 4/5\n",
      "7/7 [==============================] - 0s 10ms/step - loss: 0.4359 - accuracy: 0.7617 - val_loss: 0.5403 - val_accuracy: 0.7551\n",
      "Epoch 5/5\n",
      "7/7 [==============================] - 0s 10ms/step - loss: 0.5650 - accuracy: 0.7409 - val_loss: 0.6612 - val_accuracy: 0.7551\n"
     ]
    }
   ],
   "source": [
    "# `Sequential` provides training and inference features on this model.\n",
    "model = tf.keras.Sequential([\n",
    "  feature_layer,\n",
    "  layers.Dense(128, activation='relu'),\n",
    "  layers.Dense(128, activation='relu'),\n",
    "  layers.Dense(1)\n",
    "])\n",
    "\n",
    "# `Compile` configures the model for training.\n",
    "model.compile(optimizer='adam',\n",
    "              loss=tf.keras.losses.BinaryCrossentropy(from_logits=True),\n",
    "              metrics=['accuracy'])\n",
    "\n",
    "# `Fit` trains the model for a fixed number of epochs\n",
    "history = model.fit(train_ds,\n",
    "          validation_data=val_ds,\n",
    "          epochs=5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "GnFmMOW0Tcaa"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2/2 [==============================] - 0s 4ms/step - loss: 0.4773 - accuracy: 0.7705\n",
      "Accuracy 0.7704918\n"
     ]
    }
   ],
   "source": [
    "# `Evaluate` returns the loss value & metrics values for the model in test mode.\n",
    "loss, accuracy = model.evaluate(test_ds)\n",
    "print(\"Accuracy\", accuracy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Visualize the model loss curve\n",
    "\n",
    "Next, we will use Matplotlib to draw the model's loss curves for training and validation.  A line plot is also created showing the accuracy over the training epochs for both the train (blue) and test (orange) sets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmQAAAFNCAYAAACuWnPfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzs3Xl41PW1+PH3STJZyR7WhATCvsomCkHArUWr4q642ypqa1262t/tYrd7e+/1eq2tS9FqrxtKta6lAioICihhEZQtECAJa4AkBLIn5/fHd4JDSEJCZklmzut55pmZ73pmDB/PfFZRVYwxxhhjTOCEBToAY4wxxphQZwmZMcYYY0yAWUJmjDHGGBNglpAZY4wxxgSYJWTGGGOMMQFmCZkxxhhjTIBZQmY6DRH5m4j8ro3H7hSRCzp6HWOMaeStMsiY02EJmTHGGGNMgFlCZowxxgQREYkIdAym/SwhM+3irqb/sYisF5FjIvJXEekpIv8SkXIR+UBEkj2Ov0xEvhKRUhFZIiLDPPaNFZE17vNeA6Kb3OsSEVnnPne5iIw+zZjvFJFtInJYRN4RkT7u7SIi/ysiB0TkiIhsEJGR7n0Xi8hGd2y7ReRHp/WFGWO8qiuUQSLyLRFZ6y5XCkXk4Sb7p7ivV+ref5t7e4yI/I+I7BKRMhH5xL1tuogUNfM9XOB+/bCIvC4iL4nIEeA2EZkoIivc99grIn8WkUiP80eIyCJ3ubhfRP6fiPQSkQoRSfU4bpyIFIuIqy2f3Zw+S8jM6bgKuBAYDFwK/Av4f0B3nL+p+wBEZDAwF3jAvW8+8K6IRLoLhreAF4EU4O/u6+I+dyzwHHAXkAr8BXhHRKLaE6iInAf8B3At0BvYBbzq3v0NYKr7cyS6jznk3vdX4C5VjQdGAh+1577GGJ/q7GXQMeAWIAn4FnCPiFzuvm6WO94/uWMaA6xzn/cIMB6Y7I7pJ0BDG7+TmcDr7nu+DNQDDwJpwCTgfOC77hjigQ+A94E+wEDgQ1XdByzBKQsb3Qy8qqq1bYzDnCZLyMzp+JOq7lfV3cAy4DNVXauqVcCbwFj3cdcB/1TVRe5/zI8AMTiFzdmAC3hMVWtV9XVglcc9ZgN/UdXPVLVeVf8PqHaf1x43As+p6hpVrQZ+BkwSkX5ALRAPDAVEVTep6l73ebXAcBFJUNUSVV3TzvsaY3ynU5dBqrpEVTeoaoOqrsdJCqe5d98AfKCqc933PaSq60QkDPg2cL+q7nbfc7m73GqLFar6lvuelaq6WlVXqmqdqu7ESSgbY7gE2Keq/6OqVaparqqfuff9H3ATgIiEA7NwklbjY5aQmdOx3+N1ZTPvu7lf98GpkQJAVRuAQiDdvW+3nri6/S6P11nAD93V7aUiUgr0dZ/XHk1jOIpTC5auqh8BfwaeAA6IyBwRSXAfehVwMbBLRD4WkUntvK8xxnc6dRkkImeJyGJ3U18ZcDdOTRXua2xv5rQ0nCbT5va1RWGTGAaLyHsiss/djPnvbYgB4G2cH6P9cWohy1T189OMybSDJWTGl/bgFGqA02cLpyDYDewF0t3bGmV6vC4Efq+qSR6PWFWd28EY4nCaH3YDqOrjqjoeGI7T/PFj9/ZVqjoT6IHTrDGvnfc1xgReoMqgV4B3gL6qmgg8DTTepxAY0Mw5B4GqFvYdA2I9Pkc4TnOnJ23y/ilgMzBIVRNwmnQ9Y8huLnB3LeM8nFqym7HaMb+xhMz40jzgWyJyvrtD6A9xqvyXAyuAOuA+EXGJyJXARI9znwHudv/SFBGJc3eUjW9nDHOB20VkjLvvx7/jNG/sFJEz3dd34RR4VUCDu3/JjSKS6G7mOELb+3EYYzqPQJVB8cBhVa0SkYk4zZSNXgYuEJFrRSRCRFJFZIy79u454FER6SMi4SIyyV1ubQWi3fd3AT8HTtWXLR6n7DoqIkOBezz2vQf0FpEHRCRKROJF5CyP/S8AtwGXYQmZ31hCZnxGVbfg/Mr6E86vv0uBS1W1RlVrgCtx/tEfxunr8Q+Pc3OBO3GaFEuAbe5j2xvDB8AvgDdwfhEPAK53707AKXRLcJoqDgH/7d53M7DTXdV/N05fNGNMFxLAMui7wG9EpBz4JR417KpagNMd4ofu+64DznDv/hGwAacv22HgP4EwVS1zX/NZnNq9Y8AJoy6b8SOcRLAcp5x7zSOGcpzmyEuBfUAecK7H/k9xfoSuUVXPZlzjQ3Ji87kxxhhjQp2IfAS8oqrPBjqWUGEJmTHGGGOOE5EzgUU4feDKAx1PqLAmS2OMMcYAICL/hzNH2QOWjPmX1ZAZY4wxxgSY1ZAZY4wxxgSYJWTGGGOMMQHW5VaET0tL0379+gU6DGOMH61evfqgqjadCLNLsjLMmNDS1vKryyVk/fr1Izc3N9BhGGP8SESCZi4kK8OMCS1tLb+sydIYY4wxJsAsITPGGGOMCTBLyIwxxhhjAqzL9SFrTm1tLUVFRVRVVQU6lKARHR1NRkYGLpcr0KEYE/SsDPMuK79MVxQUCVlRURHx8fH069cPEQl0OF2eqnLo0CGKioro379/oMMxJuhZGeY9Vn6ZrioomiyrqqpITU21gsxLRITU1FT7tW6Mn1gZ5j1WfpmuKigSMsAKMi+z79MY/7J/c95j36XpioImIQuk0tJSnnzyyXafd/HFF1NaWuqDiIwxpu2sDDMm8Cwh84KWCrO6urpWz5s/fz5JSUm+CssYY9rEyjBjAi8oOvU3q6EBqkrBFQ2uWJ/e6qGHHmL79u2MGTMGl8tFdHQ0ycnJbN68ma1bt3L55ZdTWFhIVVUV999/P7Nnzwa+nrH76NGjXHTRRUyZMoXly5eTnp7O22+/TUxMjE/jNsYYsDLM56qOwM5l0HsMJKYHOhrTWalql3qMHz9em9q4ceNJ27S+XnX3WtWyopP3edmOHTt0xIgRqqq6ePFijY2N1fz8/OP7Dx06pKqqFRUVOmLECD148KCqqmZlZWlxcbHu2LFDw8PDde3ataqqes011+iLL77o87hPpdnv1ZgAAHK1E5Q/3ni0uQzzo2AswwL9nR5Xskv1z2ep/irBeTxxtuqCn6vmf6xaWx3o6IwftLX8Croasl+/+xUb9xxx3tRWAofAVdChaw7vk8CvLh3R5uMnTpx4wnDrxx9/nDfffBOAwsJC8vLySE1NPeGc/v37M2bMGADGjx/Pzp07OxSzMaZrOqEM8xIrwwJk92p45Xqoq4Yrn4XyPZC3CFY+Bcsfh8h4yJ4Ggy6EgRda7VmIC7qE7ARh4VBfAyjgv1E3cXFxx18vWbKEDz74gBUrVhAbG8v06dObHY4dFRV1/HV4eDiVlZV+idUYY5qyMswLNr4D/5gN3brDre9Cj6HO9pz7oboc8j+GbYsg7wPY/J6zr8dwGHiBk6D1PRsiIgMXv/G7oEvITvgVWHUEDm+H1IEQFe+ze8bHx1NeXt7svrKyMpKTk4mNjWXz5s2sXLnSZ3EYY7q+9tRkeYuVYV6kCsv/BIt+CRkT4Pq5TlLmKSoehl3iPFSheDPkLbTasxAXdAnZCSLdv/Kqj/o0IUtNTSUnJ4eRI0cSExNDz549j++bMWMGTz/9NMOGDWPIkCGcffbZPovDGGNOh5VhXlJfC/N/DKufh+GXwxVPg+sUAxtEoMcw53Gq2rPG5CzzbAi3ZaGCjTj9zbqOCRMmaG5u7gnbNm3axLBhw5o/oXgzSDikDfJDdMGl1e/VGD8SkdWqOiHQcXhDu8swc1r8/p1WlcHfb4PtH8GUH8B5v4CwDs4s1bT2rGAlNNRa7VkX09byK7hryAAiu8Gxg6ANIDbtmjHGGC8rLYBXroODW+GyP8O4m71zXas9CykhkpAVQ00FRHULdDTGGGOCiedIypvegOzpvrtXa33PVjwJn/7Ras+6sBBIyNz9yGqOWkJmjDHGeza9C2/cefJISn9oc+3ZCBh0gdWedQHBn5CFuyAiGmqOBToSY4wxwaAtIyn9zWrPurzgT8jAabasPOz8kYr/5iMzxhgTZOrrYP6P2jeS0t/8XHtWVVtPaUUtJRU1JMdG0isx2osfJnSESEIWBxUHnZn7I327rqUxxpggVXXEPZLyQ++NpPSHNtaeNfSfxtHMcznU6xyKw7pTWlFDaUUtpZU1lFTUHn9f0rjdva+qtuGE2w3tFc+5Q3tw7pAejMtMIiK8C3xHnUCIJGTuvmM1RztFQtatWzeOHj3Knj17uO+++3j99ddPOmb69Ok88sgjTJjQ8kjZxx57jNmzZxMb63ymiy++mFdeeYWkpCSfxW6MMSFZhp0wkvJPMO6WQEfUovoGpazSSaBKKmopq6yh5JiTSJVV1lJSUUdpxTmU1p1NdVIpA8rXMK46l5zNK0jf8h4JQFVDX9Y0jGFJwxnkNgxGwl0kxkSSHOsiOTaSvimxjM5wkRQbSVKsi6QY57mopILFm4t5Zmk+Ty3ZTkJ0BFMHd+fcIT2YNqQ7ad2iThl/qAqNhCwiEsIjnYSMHoGO5rg+ffo0W5C11WOPPcZNN910vDCbP3++t0IzxphTCpkyzJ8jKT2oKkeq6ihz10odT6iO1VBa6a6hqvCovXLvO1JV1+I1wwQSY5ykKjHWRXJCCjW9LmJzzEz2x0SQpQUMLF1Bn4OfMLv4fe5ueBeN7AYDzkUGXugs7XSKvmezpw7gSFUtn+YdZPGWAyzeUsx76/ciAqPTE4/Xno1KTyQszLoRNQqNhAycWrLqIz7pR/bQQw/Rt29fvve97wHw8MMPExERweLFiykpKaG2tpbf/e53zJw584Tzdu7cySWXXMKXX35JZWUlt99+O1988QVDhw49YR24e+65h1WrVlFZWcnVV1/Nr3/9ax5//HH27NnDueeeS1paGosXL6Zfv37k5uaSlpbGo48+ynPPPQfAHXfcwQMPPMDOnTu56KKLmDJlCsuXLyc9PZ23336bmJhO1v/BGONXVoa1wksjKStr6jl0rPqEpr6SilrKjidUjcnV18lWWWUt9Q0tT94eHx1BcmMNVWwk/dLiSIr5utbqeNIV69RsJcVEEh8dcYokaDBwgfPS3fdMGvuebXrX2d6GvmcJ0S4uGtWbi0b1pqFB2bj3CIs3H2DxlgP88cM8Hvsgj7RukUwb3INzh3bnnEHdSYwJ7RGgPkvIROQ54BLggKqObOW4M4EVwPWqevo/tU4lyt2xv64aXN7tcHjdddfxwAMPHC/M5s2bx4IFC7jvvvtISEjg4MGDnH322Vx22WVIC8ngU089RWxsLJs2bWL9+vWMGzfu+L7f//73pKSkUF9fz/nnn8/69eu57777ePTRR1m8eDFpaWknXGv16tU8//zzfPbZZ6gqZ511FtOmTSM5OZm8vDzmzp3LM888w7XXXssbb7zBTTfd5NXvwxjTtVgZ1gzPkZTp42HWXOh2ei0s81YV8rM3N7SYXMVGhjvJU4yL5DgXvRNjjidUjclWkntf4+vEGJfv+2a1deTmgOlOctZC7VlYmDAyPZGR6Yl8//xBHD5Ww9KtxSzecoAPN+/njTVFhIcJ47OSOXeIk6AN6Rnf4t9asPJlDdnfgD8DL7R0gIiEA/8JLPTaXf/1EOzbcPJ2bYDaY84UGGHtzMJ7jYKL/tDi7rFjx3LgwAH27NlDcXExycnJ9OrViwcffJClS5cSFhbG7t272b9/P7169Wr2GkuXLuW+++4DYPTo0YwePfr4vnnz5jFnzhzq6urYu3cvGzduPGF/U5988glXXHEFcXHOHGxXXnkly5Yt47LLLqN///6MGTMGgPHjx7Nz5872fRfGGN9qqQzrCCvD2seLIyk3FJXx87e+ZHxWMleNSz/eDyvJXWuVGOsiKiLcyx/AB041crOx9iwmGWg9kUoBLnc/NArqXUpNfQO1+5T6PQ2wGMpEcIWH4Qp3njtdanbBwzD+Vq9e0mcJmaouFZF+pzjs+8AbwJm+iuM4ESAMGurbn5C1wTXXXMPrr7/Ovn37uO6663j55ZcpLi5m9erVuFwu+vXrR1VVVbuvu2PHDh555BFWrVpFcnIyt91222ldp1FU1NcdKsPDw09oVjAmFInIDOCPQDjwrKr+ocn+/wXOdb+NBXqoapJ7Xz3QmD0VqOpl/ona+6wMc/PiSMqyilq++8pqUrtF8vRN40mJi/RurIHUbO3ZImfwQzsITiLSmIwcq6lnT2kFRSWV7C2rora6gXAReiZGk54UQ0ZyDAnRnaBpM3WA1y8ZsD5kIpIOXIFT0HkvIWvlVyCHdzgTxPYc4fV+ZNdddx133nknBw8e5OOPP2bevHn06NEDl8vF4sWL2bVrV6vnT506lVdeeYXzzjuPL7/8kvXr1wNw5MgR4uLiSExMZP/+/fzrX/9i+vTpAMTHx1NeXn5Sdf8555zDbbfdxkMPPYSq8uabb/Liiy969fMaEwzctfRPABcCRcAqEXlHVTc2HqOqD3oc/31grMclKlV1jFeDaq0M8yErw4DSQnjlWq+MpGxoUH7493XsLa1i3t2TgisZa8qz9qyD4oBB7kdNXQO5Ow+zeMsB/ralmG3bjgLQLzX2+MCAif1TiHZ1gRrGNghkp/7HgJ+qasOp2olFZDYwGyAzM/P07xjVDapKob4GIrw79HbEiBGUl5eTnp5O7969ufHGG7n00ksZNWoUEyZMYOjQ1juC3nPPPdx+++0MGzaMYcOGMX78eADOOOMMxo4dy9ChQ+nbty85OTnHz5k9ezYzZsygT58+LF68+Pj2cePGcdtttzFx4kTA6RA7duxYa5405mQTgW2qmg8gIq8CM4GNLRw/C/iVn2Lzq5Avw3avgbnXQ22VV0ZSzlmWzwebDvCrS4czLjPZKyGGmsiIMCYPTGPywDT+7VtQcKiCJVsPsHjzAV75rIDnP91JjCucnIFpnDu0O9OH9CA9qesOUhPVlkdwdPjiTpPle8116heRHXzd0JwGVACzVfWt1q45YcIEzc3NPWHbpk2bGDasDZl5baVTrZqUCbGpbfkIIa3N36sxPiYiq1W15QmtTv+6VwMzVPUO9/ubgbNU9d5mjs0CVgIZqlrv3lYHrAPqgD+cqvyCDpZhps3a9Z16jqS84e8dXpNyZf4hbnz2M2aM6MWfbxgbcp3T/aGypp6V+YdYvOUAH20+QFGJ03Q9pGfjpLTdGZeVjKsTTErb1vIrYDVkqtq/8bWI/A0ncTtlYdYhEdEg4c58ZJaQGWPa53rg9cZkzC1LVXeLSDbwkYhsUNXtTU/0Wi2/8S5VWPFnWPiLDo+kbHSgvIrvz11LVkosf7hqlCVjPhITGe4kXkN78OvLlO3FR1m82Rm5+eyyfJ7+eDvxnpPSDu5O9/jOPSmtL6e9mAtMB9JEpAinmt8FoKpP++q+pwjKPR+ZLTRujAFgN9DX432Ge1tzrge+57lBVXe7n/NFZAlO/7KTEjJVnQPMAaeGrMNRm47zwZqUdfUN3Dd3LeVVtbz4nYnEd4bO5yFARBjYI56BPeK5c2o25VW1fLrt4PEE7Z/r9wJwRkYi04c4SdzoTjgprS9HWc5qx7G3+SqOk0R1g+oypx9ZeBB3sjTGtMUqYJCI9MdJxK4Hbmh6kIgMBZJx5kxs3JYMVKhqtYikATnAf/klatMxJ4ykfBDO+6VX1qR8dNFWVuYf5n+uOYOhvRI6Hqc5LfHRLmaM7M2Mkb1RVb7ac4Ql7qbNxz/K448f5pEaF8m0IU7t2dRB3UmMDXzyHDQz9atq26qGI515bag5BjGWkLXEl30LjeksVLVORO4FFuBMe/Gcqn4lIr8BclX1Hfeh1wOv6on/MIYBfxGRBiAMpw9ZS4MB2hKLNW95SavlV2mhe03KLV5dk/LDTft5csl2Zk3sy1XjM7xyTdNxIl9PSnvvec6ktMvyivlos5Og/WPNbsLDhHGZScdHbg7tFZhJaYMiIYuOjubQoUOkpqae+kt0xYKEQfVR9wR2pilV5dChQ0RHe3dFA2M6I1WdD8xvsu2XTd4/3Mx5y4FR3oihXWWYaVWr5ZeXR1I2KjxcwYOvrWNEnwR+dekIr1zT+EZKXCQzx6Qzc0w69Q3KusLS47Vn//X+Fv7r/S30Tox2mjaHdCdnYBpxUf5JlYIiIcvIyKCoqIji4uK2nXC0DPQwxB/1bWBdWHR0NBkZ9ivPGH9odxlmWtVs+eU5kvKWdzo8krJRdV093315DQo8eeO4oJkTKxQ0Ltc0PiuZH35jCPuPVPHxFqf27N0v9jD38wIiw8M4KzvleIKW3b2bz+IJioTM5XLRv3//Ux/YaOk/4aPfwk92QGyK7wIzxpg2aHcZZtrOByMpPf32vY1s2F3GnJvHk5Ua57XrGv/rmRDNtWf25doz+zqT0u467F4QvZjfvreR377nTEo7fUgPrh6fwcj0RK/ePygSsnbLck9MWLAChn4rsLEYY4zxjRNGUs6EK/7S4ZGUnt5au5uXVhZw19RsvjGi+TU+TdcUGRHG5AFpTB7gTEpbeLjieNPm3M8LGN47wRIyr0gfB+FRsGu5JWTGGBOMfDSSstHW/eX87B8bmNgvhR99c4jXrms6p74psdw8qR83T+pHVW09vhj3FpoJWUQUZExwEjJjjDHBxUcjKRsdra7j7pdWExcVwZ9uGNspZoM3/uOrfoKh+1eUNRn2fgHV5YGOxBhjjLfsXgPPng9lRXDj615PxlSVn/1jAzsPHuPxWWPomWCj0Y13hHZCpvVQ+HmgIzHGGOMNm96D5y92WkG+sxAGnOv1W7y4chfvfrGHH35jCJMHpHn9+iZ0hW5CljHRWdfSmi2NMaZrU4Xlf4LXboKeI+COD702rYWndYWl/Pa9jZw3tAf3TBvg9eub0BaafcjAWUKpzxhLyIwxpiurr4N//Rhyn/PJSMpGJcdq+N7La+gRH82j157R6dZBNF1f6NaQgdNsuTvXmbXZGGNM11J1BF651knGpjwIV//NJ8lYQ4Py4Lx1FJdX89RN40iKtWX3jPeFeEKW4ywyvnt1oCMxxhjTHqWF8NwM2PExXPo4XPCwV6e18PTkkm0s2VLMLy4dzuiMJJ/cw5jQTsgyzwbEmi2NMaYraTqScvytPrvVp9sO8uiircwc04ebzsr02X2MCe2ELCbZ6QC669NAR2KMMaYt/DCSstG+siruf3Ut2d278e9XjLKF341PhXZCBk4/ssLPob420JEYY4xpiSos/7PPR1I2qq1v4Ptz11BRU8/TN40jLip0x8AZ/7CELGsy1B6DvesDHYkxxpjm1NfBP38AC/8Nhl8Gt73n1QXCm/PfC7awamcJ/3HlKAb2iPfpvYwBS8ggc7LzbM2WxhjT+fhpJKWn97/cx5yl+dx8dhYzx6T79F7GNLKELL4npA6EghWBjsQYY4wnP46kbLTr0DF+/PcvGJ2RyM8vGebTexnjyRrFATInwaZ3oaHB5//YjTHGtMHuNTD3eqitdEZS+rDzfqOq2nrueWkNYWHCEzeMIyrCN4tIG9Mcyz7AmY+sqhSKNwU6EmOMMY0jKcN9P5LS08PvfMXGvUf43+vOoG9KrF/uaUwjS8jA6dgPNh+ZMcYEUtORlHd+CD3802z499xCXl1VyPfOHcB5Q3v65Z7GeArahKysspbf/3MjuTsPn/rgpExIyLCO/cYYEygBGEnZaNPeI/zi7S+ZlJ3KgxcM9ss9jWkqaBOyqIgw3lizm6eWbD/1wSJOLdmu5c4vNGOMMf5TdQTmXueMpMx5wC8jKRuVV9Xy3ZfXkBDt4o+zxhARHrT/WzSdXND+5UW7wrllUhYfbj5A3v7yU5+QNRmO7ofD+b4PzhhjjKNxJGX+Emck5YW/9tvgKlXlp2+sp+BwBX+aNZYe8dF+ua8xzQnahAzglkn9iHaF8cyyNiRZWTnOszVbGmOMf9TVwPMXQVmhz9ekbM7zn+5k/oZ9/OSbQzgrO9Wv9zamqaBOyFLiIrl2Ql/eWruHA0eqWj84bRDEplnHfmOM8ZeC5U4yNvPPfhtJ2Wj1rhL+ff4mLhzek9lTs/16b2OaE9QJGcAdU7Kpa2jg+eU7Wz/weD8yqyEzxhi/yFsE4ZEw8AK/3vbQ0WrufWUNvZOieeSaM2zRcNMpBH1Clpkay0Uje/PSyl0cra5r/eCsHCgtcPo0GGOM8a2tC6DfFIiM89st6xuUB15bx6FjNTx143gSY1x+u7cxrQn6hAxg9tRsyqvqePXzgtYPbJyPzJZRMsYY3zqcD4fyYNA3/XrbP32Ux7K8g/z6shGMTE/0672NaU1IJGRn9E3irP4pPPfJDmrrG1o+sOcIiEq0ZktjjPG1vEXO86AL/XbLpVuL+eOHeVw5Lp3rz+zrt/sa0xYhkZAB3DUtmz1lVby3fk/LB4WFQ+bZsMtqyIwxxqe2LoDUgZA6wC+321Nayf2vrmVwj3h+d/lI6zdmOp2QScimD+7BoB7d+MvH+Whrk79mTYKDW+Bosf+CM8aYUFJzDHZ+4rfmypq6Bu59ZQ01dQ08edM4YiMj/HJfY9ojZBKysDDhzqnZbN5XzrK8gy0f2DgfmfUjM8YY39ixFOqrYfA3/HK7P/xrM2sKSvnPq0czoHs3v9zTmPYKmYQMYOaYPvSIj2LO0lYmiu09BiJibD4yY0KEiMwQkS0isk1EHmpm//+KyDr3Y6uIlHrsu1VE8twP/85q2pVtXQCR3SBzss9vNX/DXp77dAe3Te7HJaP7+Px+xpyukErIoiLCuT2nP59sO8iXu8uaPygiEvqeaR37jQkBIhIOPAFcBAwHZonIcM9jVPVBVR2jqmOAPwH/cJ+bAvwKOAuYCPxKRJL9GX+XpAp5CyF7ulPe+lB+8VF+8vp6xmYm8f8uHubTexnTUSGVkAHccFYmcZHhrS+nlJUD+zZAVQtJmzEmWEwEtqlqvqrWAK8CM1s5fhYw1/36m8AiVT2sqiXAImCGT6MNBgc2wpHdMMi3zZWVNfV89+U1uMKFJ24YR2REyP3vznQxIfcXmhjjYtbETN5bv5eikormD8qaDCgUfObX2IwxfpcOeM4EXeTedhIRyQL6Ax+dxrmzRSRXRHKLi0N8wNDWBc6zDxMyVeXnb33Jlv3lPHb9WPokxfjsXsZ4S8hO/sMfAAAgAElEQVQlZADfntIfAZ77ZGfzB6RPgDCXNVsaYzxdD7yuqvXtPVFV56jqBFWd0L17dx+E1oXkLYReoyGht89uMS+3kDfWFHHfeYOYNjjEv2/TZYRkQtYnKYZLz+jDq6sKKKuoPfmAyFhIH2cd+40JfrsBzxlCM9zbmnM9XzdXtvdcA1BZAoWf+bR27Ks9Zfzi7a84Z1Aa950/yGf3McbbQjIhA7jznGwqaup56bNdzR+QNRn2rIGaFpo1jTHBYBUwSET6i0gkTtL1TtODRGQokAx4zoezAPiGiCS7O/N/w73NtGTbh6ANMNg384+VVdby3ZfXkBIbyWPXjSE8zCZ/NV2HzxIyEXlORA6IyJct7L9RRNaLyAYRWS4iZ/gqluYM75PAOYPS+NvynVTXNdMCkZUDDXVQtMqfYRlj/EhV64B7cRKpTcA8Vf1KRH4jIpd5HHo98Kp6zCqtqoeB3+IkdauA37i3mZbkLYSYFEgf7/VLqyo//vsX7C6p5Ikbx5LaLcrr9zDGl3xZQ/Y3Wh9xtAOYpqqjcAq1OT6MpVl3TR1AcXk1b61tppWh70SQMGu2NCbIqep8VR2sqgNU9ffubb9U1Xc8jnlYVU+ao0xVn1PVge7H8/6Mu8tpqIdtH8DAC5xl6rzs2WU7WLhxPz+7eBjjs1K8fn1jfM1nCZmqLgVa/LWoqsvdQ8UBVuL0v/CrnIGpDO+dwJyl+TQ0NFlOKToReo2yjv3GGOMNu9dAxSGfNFeu2nmYP7y/mYtG9uLbOf28fn1j/KGz9CH7DvAvf99URLhrWjbbi4/x0eYDJx+QleM0WdbV+Ds0Y4wJLnkLnVaHAed59bLF5dV87+U1ZKbE8l9Xj7ZFw02XFfCETETOxUnIftrKMT6bw+dbo3qTnhTT/HJKmZOgrgr2rvPqPY0xJuTkLYCMiRDrvebE+gbl/lfXUlZZy5M3jiM+2uW1axvjbwFNyERkNPAsMFNVD7V0nC/n8IkID+M7U/rz+c7DrC0oOXFnlnudNWu2NMaY01e+D/Z+4fXFxB/7YCvLtx/id5ePZFjvBK9e2xh/C1hCJiKZOGvC3ayqWwMVB8B1Z/YlMcZ1ci1ZXBqkDbGO/cYY0xF5i5znQd7rP7Z4ywH+9NE2rpvQl2sm9D31CcZ0cr6c9mIuzpw9Q0SkSES+IyJ3i8jd7kN+CaQCT4rIOhHJ9VUspxIXFcFNZ2fy/lf72HHw2Ik7syZDwUpnhJAxxpj2y1sACenQc4RXLldUUsGDr61jWO8Efj3TO9c0JtB8Ocpylqr2VlWXqmao6l9V9WlVfdq9/w5VTVbVMe7HBF/F0ha3Tu6HKyyMZ5suOp6VA9VHYH+z06kZY4xpTV0NbF8Cgy4EL3S4r66r53svr6G+XnnqxnFEu7w/hYYxgRDwTv2dRY/4aK4cl87rq4s4eLT66x1Zk5xna7Y0xpj2K1gBNeVea678/T838UVRGf99zRn0S4vzyjWN6QwsIfNwxznZVNc18MIKj+WUEjMgKcs69htjzOnIWwjhkdB/aocv9fa63bywYhd3ntOfGSN7eSE4YzoPS8g8DOzRjQuG9eTFFTuprPHoM5aV49SQqbZ4rjHGmGZsXQD9pkBUtw5dZtuBcn72jw1MyErmJzOGeik4YzoPS8iauGtaNiUVtfx9deHXG7MmOzNMHwzoYFBjjOlaDufDobwON1ceq67j7pfWEOMK5883jMMVbv/rMsHH/qqbmJCVzNjMJJ5dtoP6xuWUbD4yY4xpv+PTXVx42pdQVf7tzQ1sLz7K47PG0isx2kvBGdO5WELWhIhw19RsCg5X8P6X+5yNKdnQrZd17DfGmPbYugBSB0LqgNO+xMufFfDWuj384ILB5AxM82JwxnQulpA148LhveiXGsucpdtRVWeodtZk2Pmp9SMzxpi2qDkGOz/pUHPl+qJSfvPuRqYP6c73zh3oxeCM6XwsIWtGeJhwxznZfFFUxmc7DjsbsyZD+R4o3dX6ycYYY2DHUqivPu3mytKKGu55aQ1p3SL532vHEBZmi4ab4GYJWQuuHp9Balzk18spHe9HtiJwQRljTFexdQFEdnNGqbdTQ4Pyw3lfcKC8iiduHEdyXKQPAjSmc7GErAXRrnBumdSPjzYfIG9/OXQfBtFJ1rHfGGNORdXp0J89HSLan0w9vXQ7H24+wM+/NZyxmcleD8+YzsgSslbcPCmLaFeYU0sWFubUklnHfmOMad2BjXCkCAZ9o92nrth+iEcWbOGS0b25ZVKWD4IzpnOyhKwVKXGRXDuhL2+t283+I1VOQnZ4O5TvC3RoxhjTeW1d4Dy3MyE7cKSK789dS7+0OP5w1WjEC2tfGtNVWEJ2CndMyaa+QXn+050e/cislswYY1qUtwh6jYKE3m0+pa6+gXvnruVYdR1P3zSeblERPgzQmM7HErJTyEyN5aJRvXn5s10cTRkBrjhLyIwxpiWVJVD4Wbunu3hk4VY+33GY/7hyFIN7xvsoOGM6L0vI2uCuqdmUV9Xxau4eyDzLEjJjjGnJ9o9A62Fw2xOyRRv38/TH27nxrEwuH5vuw+CM6bwsIWuD0RlJnJ2dwl8/2UF930lw4CuoOBzosIwxpvPZuhBiUiB9fJsOLzhUwQ/nrWNUeiK/uGS4j4MzpvOyRvo2umvqAG7/2yo+qR3CNICClTD04kCHZYwxnUdDPWxbBAMvoFaF0vJqyiprKKmopbSilpKKGsrcz6WVtZRW1PBFYRkAT944jmhXeIA/gDGBYwlZG00f0p3BPbvxyJe1TA2PQnZ9agmZMSbo1TcoRyprKa1sklBVOAlVSUXt8eSqR9kG/qfiED9Z34t5q/7V4jXDw4TkWBeJMS7Sk2K4/4LR9E2J9eOnMqbzsYSsjUSEO8/J5sevr6csczRJ1o/MGNOFqCrl1XUnJFQlFTWUVdZScszjtUeyVVpZS1llbYtL+IpAYoyLpBgXSbGRXC1raCCM1DEX82B8GslxTtKVHBtJUuzXz92iImxKC2OasISsHWaOSeeRhVtYXDmIK46+BtXlEGWjgYwx/qOqVNbWN2kC9Eyuvm4OLPWszaqspb6hhcwKiI+KINEjaeqbEkty7NfJlmdClRQbSXKsi/hoF+Gea0z+5WeQMJGfXjHJD9+EMcHFErJ2iIwI4/ac/ryxIIsrIuuh8HMYeH6gwzLGBKF5uYWs2VXiUWP1dd+rmrqGFs+LcYU7zYHupGlIr3gnoYo5OaFqfJ0Y48IV3sExXuX7YO8XcP4vO3YdY0KUJWTtdMNZmfz1o2HUE0Z4wQpLyIwxPpG78zBLthQfT5qyUmMZ0zep2YSqsfYqMcYVuI7xeYuc59NYLskYYwlZuyVEu7h84iC+/Kw/Q7cvI+q8QEdkjAlG/3X1GYEOoX3yFkBCOvQcGehIjOmSbB6y03B7Tn9W6VDC96yB2qpAh2OMMYFVVwPbl8CgC52e/saYdrOE7DT0SYpBsnKI0BqO5n8e6HCMMR0gIjNEZIuIbBORh1o45loR2SgiX4nIKx7b60Vknfvxjv+i7mQKVkBNuTVXGtMBlpCdpinnXwLAlytanmvHGNO5iUg48ARwETAcmCUiw5scMwj4GZCjqiOABzx2V6rqGPfjMn/F3enkLYTwSOg/LdCRGNNlWUJ2mob0z6LA1R/d9SlVtfWBDscYc3omAttUNV9Va4BXgZlNjrkTeEJVSwBU9YCfY+z88hZCvykQ1S3QkRjTZVlC1gGu/lMY3bCZt1fvCnQoxpjTkw4Uerwvcm/zNBgYLCKfishKEZnhsS9aRHLd2y/3dbCd0uEdcHCrNVca00GWkHVAr9HnESfVLFn6IQ2tTLhojOnSIoBBwHRgFvCMiCS592Wp6gTgBuAxERnQ3AVEZLY7ccstLi72R8z+k7fQebaEzJgOsYSsAyRrMgDpZWv5cLO1YhjTBe0G+nq8z3Bv81QEvKOqtaq6A9iKk6Chqrvdz/nAEmBsczdR1TmqOkFVJ3Tv3t27nyDQ8hZC6kBIbTYXNca0kSVkHRHfC00ZwLSorcxZuj3Q0RgT0kTkHyLyLRFpT7m2ChgkIv1FJBK4Hmg6WvItnNoxRCQNpwkzX0SSRSTKY3sOsLGDH6NrqTkGO5ZZ7ZgxXmAJWQdJ1mQmhm0hd+ch1hSUBDocY0LZkzhNh3ki8gcRGXKqE1S1DrgXWABsAuap6lci8hsRaRw1uQA4JCIbgcXAj1X1EDAMyBWRL9zb/6CqoZWQ7VgK9dWWkBnjBTZTf0dl5RC19kXGRe9jzsf5PH3z+EBHZExIUtUPgA9EJBGnr9cHIlIIPAO8pKq1LZw3H5jfZNsvPV4r8AP3w/OY5cAor36IriZvIUR2A3f3DWPM6bMaso5yF0T39N/Pgo37yC8+GuCAjAldIpIK3AbcAawF/giMAxYFMKzgpApbF0L2dIiICnQ0xnR5lpB1VFImJKQzxbUVV3gYz36yI9ARGROSRORNYBkQC1yqqpep6muq+n3AJsjytgOb4EiRNVca4yWWkHWUCGRNJnrPZ1w1tg+vry7i4NHqQEdlTCh6XFWHq+p/qOpezx3uqSmMN+UtcJ4tITPGKywh84asyXB0H3ePDqO2voEXlu8MdETGhKLhHvOD4R4F+d1ABhTUti6EXqMgoXegIzEmKFhC5g1ZOc5T+TouGNaTF1buoqKmLsBBGRNy7lTV0sY37qWO7gxgPMGrsgQKP4NB3wx0JMYEDUvIvCFtMMSmwq7l3DU1m9KKWv6eWxToqIwJNeEiIo1v3AuHRwYwnuC1/SPQemuuNMaLLCHzBnc/MnZ9yoR+KYzLTOLZT/Kpq28IdGTGhJL3gddE5HwROR+Y695mvG3rQohJgQzrmmeMt1hC5i1ZOVC6C8qKmD11AIWHK3n/q32BjsqYUPJTnAla73E/PgR+EtCIglFDA2xbBAMvgLDwQEdjTNDwWUImIs+JyAER+bKF/SIij4vINhFZLyLjfBWLXzROjLhrBRcO70n/tDjmLM3HmVPSGONrqtqgqk+p6tXux19UtT7QcQWdPWug4pA1VxrjZb6sIfsbMKOV/RfhLNA7CJgNPOXDWHyv50iISoBdnxIeJtxxTn/WF5WxMv9woCMzJiSIyCAReV1ENopIfuMj0HEFna0LQMJg4PmBjsSYoNKmhExE7heRBHet1l9FZI2ItPrzSFWXAq1lIzOBF9SxEkgSka47fjosHDLPhl3LAbhqXAapcZG26Lgx/vM8zg+7OuBc4AXgpYBGFIzyFkLGRIhNCXQkxgSVttaQfVtVjwDfAJKBm4E/dPDe6UChx/si97auK2syHNwCR4uJdoVz6+R+LN5SzNb95YGOzJhQEKOqHwKiqrtU9WHgWwGOKbiU74O962DQhYGOxJig09aErHEo+cXAi6r6lcc2nxOR2SKSKyK5xcXF/rpt+7nnI6NgBQA3n51FjCucOUut1cQYP6gWkTAgT0TuFZErsCWTvCvPvSToYJt/zBhva2tCtlpEFuIkZAtEJB7o6JwOu4G+Hu8z3NtOoqpzVHWCqk7o3r17B2/rQ73HQETM8WbL5LhIrp2QwdvrdrOvrCrAwRkT9O7HWcfyPmA8cBNwa0AjCjZ5CyG+j9Nn1hjjVW1NyL4DPAScqaoVgAu4vYP3fge4xd0v7WygrOn6c11ORKQzL8+uT49vuuOcbOoblOeX26LjxviKexLY61T1qKoWqertqnqVu3+q8Ya6Gti+2GmuFL81kBgTMtqakE0CtqhqqYjcBPwcKGvtBBGZC6wAhohIkYh8R0TuFpG73YfMB/KBbcAzQHCsOZeVA/u/hCrn6+mbEstFo3rzysoCyqtqAxycMcHJPb3FlEDHEdQKVkBNuTVXGuMjEW087ingDBE5A/gh8CzOCKZpLZ2gqrNau6A6E3R9r4337zqyJoM2QOHnxzu+3jU1m3+u38urnxdy59TsAAdoTNBaKyLvAH8HjjVuVNV/BC6kIJK3EMIjoX+Lxb4xpgPaWkNW506gZgJ/VtUngHjfhdWFZZwJYREnNFuOzkhiUnYqf/1kBzV1tpySMT4SDRwCzgMudT8uCWhEwSRvodMCEGXjJIzxhbbWkJWLyM9wprs4xz2SyeW7sLqwyFjoM+54x/5Gs6dlc/vzq3j3iz1cNT4jQMEZE7xUtaP9Wk1LDu+Ag1thwrcDHYkxQautCdl1wA0485HtE5FM4L99F1YXlzUZVjwBNRVOggZMH9ydIT3jeWZZPleOS0esU6wxXiUizwMnrVWmqpZFdFTjdBe2XJIxPtOmJktV3Qe8DCSKyCVAlaq+4NPIurKsHGiohd25xzeJCHdOzWbzvnI+3tqJ51Izput6D/in+/EhkAAcDWhEwSJvAaQMgNQBgY7EmKDV1qWTrgU+B64BrgU+E5GrfRlYl5Z5FiAnNVtedkYfeiVE20SxxviAqr7h8XgZp6yaEOi4uryaCtixzEZXGuNjbe3U/284c5Ddqqq3ABOBX/gurC4uOhF6jTqhYz9AZEQYt+f0Y/n2Q3y5u9VZQ4wxHTcI6BHoILq8HUuhvtqaK43xsbYmZGGqesDj/aF2nBuasnKgcJUzmaKHWWdl0i0qgr9YLZkxXiUi5SJypPEBvAv8NNBxdXl5C8AV5/SNNcb4TFuTqvdFZIGI3CYit+H00Zjvu7CCQNZkqKt0FuL1kBDt4oazMpm/YS+FhysCFJwxwUdV41U1weMxWFXfCHRcXZqq06F/wLkQERXoaIwJam3t1P9jYA4w2v2Yo6r2y7M1jb8mmzRbAtye0w8B/vqJLadkjLeIyBUikujxPklELg9kTF3egU1QVmjNlcb4QZubHd0dZX/gfrzpy6CCQlwapA05qWM/QO/EGC4b04fXVhVSWlHTzMnGmNPwK1U93jlTVUuBXwUwnq4vb4Hz7F51xBjjO60mZE37ZHg8yt19NExrsiZBwUpoqD9p1+yp2VTW1vPSyl0BCMyYoNRcedbWuRZNc/IWOQOUEvoEOhJjgl6rCVkzfTIaH/GqmuCvILusrByoPgL7vzpp19BeCUwb3J2/Ld9FVe3JCZsxpt1yReRRERngfjwKrA50UF1WZYnzg3KQTXdhjD/YSElfOt6P7ORmS3AWHT94tJo31+72Y1DGBK3vAzXAa8CrQBXwvYBG1JVt/wi03vqPGeMnlpD5UmIGJGU227EfYNKAVEamJ/DMsnwaGk5a8cUY0w6qekxVH1LVCap6pqr+P1U9Fui4uqy8RRCTAhk2t64x/mAJma9l5Tg1ZHpywiUizJ46gPziY3ywaX8AgjMmeIjIIhFJ8nifLCIL2nDeDBHZIiLbROShFo65VkQ2ishXIvKKx/ZbRSTP/bjVO5+kE2hocBKygRdAWHigozEmJFhC5mtZk6HiIBzMa3b3xSN7kZEcY8spGdNxae6RlQCoagmnmKlfRMKBJ4CLgOHALBEZ3uSYQcDPgBxVHQE84N6egjOK8yyc1Ut+JSLJ3vs4AbRnjVNuWXOlMX5jCZmvZeU4zy00W0aEh/GdKf3J3VXC6l0lfgzMmKDTICKZjW9EpB9wqr4AE4FtqpqvqjU4fc9mNjnmTuAJd4KHx6ol3wQWqeph975FwIwOf4rOIG8hSBgMPD/QkRgTMiwh87WUbOjWs8WO/QDXTuhLYoyLOUu3+zEwY4LOvwGfiMiLIvIS8DFOzVZr0oFCj/dF7m2eBgODReRTEVkpIjPacW7XtHUBZJwJsSmBjsSYkGEJma+JOM2Wuz5tth8ZQFxUBLdMymLhxv3kFx/1c4DGBAdVfR+YAGwB5gI/BCq9cOkInIXKpwOzgGc8+6q1hYjMFpFcEcktLi72Qkg+VL7PWfLNmiuN8StLyPwhKweO7IbSghYPuWVSP1zhYTyzzJZTMuZ0iMgdwIc4idiPgBeBh09x2m6gr8f7DPc2T0XAO6paq6o7gK04CVpbzgVAVee4R39O6N69e9s+UKBs+8B5HmzzjxnjT5aQ+cMp5iMD6B4fxVXjMnhjTRHF5dV+CsyYoHI/cCawS1XPBcYCpa2fwipgkIj0F5FI4HrgnSbHvIVTO4aIpOE0YeYDC4BvuEdzJgPfcG/r2rYugPg+0HNkoCMxJqRYQuYP3YdBdFKLHfsb3XlOf2rrG3hhxU6/hGVMkKlS1SoAEYlS1c3AkNZOUNU64F6cRGoTME9VvxKR34jIZe7DFgCHRGQjsBj4saoeUtXDwG9xkrpVwG/c27qu+lrYvthZu1Ik0NEYE1JsnTd/CAtz9yNruYYMILt7Ny4c1pMXV+7inukDiI20/zzGtEORu2/XW8AiESkBTrlYrKrOB+Y32fZLj9cK/MD9aHruc8BzHYy78yhYATXl1lxpTABYDZm/ZE6Cw9uhvPUJYO+alk1pRS3zVhW2epwx5kSqeoWqlqrqw8AvgL8Clwc2qi5m6wIIj4T+0wIdiTEhxxIyf2mcj6yg9Vqy8VkpjM9K5tlPdlBX3+CHwIwJPqr6saq+455bzLRV3iKnrIrqFuhIjAk5lpD5S+/R4Io7ZbMlwOyp2RSVVPKvL/f5ITBjjAFKdsLBLdZcaUyAWELmL+Eu6DuxTQnZhcN6kp0Wx5yl+WgLc5cZY4xXbV3oPNv8Y8YEhCVk/pSVA/u/gorWB2KFhQl3nJPNht1lrMg/5KfgjDEhLW8hpAyA1AGBjsSYkGQJmT9lTQYUCj875aFXjksnrVukLTpujPG9mgrYucyaK40JIEvI/Cl9vDOC6RTzkQFEu8K5dVI/lmwpZsu+cj8EZ4wJWTuWQl2VM/+YMSYgLCHzJ1c0pE9oUz8ygJvOziLGFW61ZMYY38pb6Aw6ahwNbozxO0vI/C1rMuxZB9WnXkQ8OS6S687syztf7GZfWZUfgjPGhBxVJyEbcC5ERAU6GmNCliVk/pY1GbQeij5v0+HfmdKf+gbl+U9t0XFjjA8c2ARlhdZcaUyAWULmb30ngoS3udmyb0osF4/qzSufFVBeVevj4IwxISfPprswpjOwhMzfouKh9xltTsgA7po6gPLqOuZ+XuDDwIwxISlvIfQaBQl9Ah2JMSHNErJAyJoMRblQ27Z+YaMyEpmUncpzn+ykps6WUzLGeEllCRSstNoxYzoBS8gCIWsy1FfDnjVtPmX2tGz2HaninS/2+DAwY0xI2b7Y6dM6yOYfMybQLCELhMxJznMb5iNrNH1wd4b0jOcZW07JGOMteQshJhkyJgQ6EmNCniVkgRCbAj2Gw64VbT5FRJg9NZst+8tZsrXYh8EZY0JCQwPkLYKBF0BYeKCjMSbkWUIWKFmTnSWU6uvafMqlZ/ShV0I0cz62iWKNMR20Zy1UHLTmSmM6CUvIAiVrMtQchX3r23xKZEQY357SjxX5h9hQVObD4IwxQS9vAUgYDDw/0JEYY7CELHAyJzvP7Zj+AmDWxEzioyL4y9LtPgjKGBMy8hZCxplOFwpjTMD5NCETkRkiskVEtonIQ83szxSRxSKyVkTWi8jFvoynU0noDSnZ7U7I4qNd3HBWJvM37KXwcIWPgjPGBLXy/U6TpU13YUyn4bOETETCgSeAi4DhwCwRGd7ksJ8D81R1LHA98KSv4umUsiZDwXKnc2073J7Tn/Aw4a+f2HJKxpjTsG2R82wJmTGdhi9ryCYC21Q1X1VrgFeBmU2OUSDB/ToRCK1JtrJynIkZize367ReidFcdkY6r60qpORYjY+CM8YErbyFEN/HmaHfGNMp+DIhSwcKPd4Xubd5ehi4SUSKgPnA930YT+eT1diPrO3zkTWaPTWbytp6Xlq5y8tBGWOCWn2tMyHsoAtBJNDRGGPcAt2pfxbwN1XNAC4GXhSRk2ISkdkikisiucXFQTQHV1IWJKS3ux8ZwJBe8Uwf0p3/W7GTqtp678dmjAlOBSug+og1VxrTyfgyIdsN9PV4n+He5uk7wDwAVV0BRANpTS+kqnNUdYKqTujevbuPwg0AEaeWbNdyOI3Z92dPzebg0Rr+sabp12qMMS3IWwjhkZA9PdCRGGM8+DIhWwUMEpH+IhKJ02n/nSbHFADnA4jIMJyELIiqwNogazIc3QeH2z/Z66TsVEalJ/Lssnzq6m3RcWNMG2xd6PRfjeoW6EiMMR58lpCpah1wL7AA2IQzmvIrEfmNiFzmPuyHwJ0i8gUwF7hNQ22hxtOcjwyc5ZTuPW8g+QeP8Z/vt29ggDEmBJXshINbrLnSmE4owpcXV9X5OJ31Pbf90uP1RiDHlzF0et2HQGyqk5CNu7ndp39zRC9unZTFM8t2MLRXAleNz/BBkMaYoJDnnu5isC2XZExnE+hO/UYEMic585Gdpp9fMpxJ2an87M0NrCss9WJwxpigsnUBpAyA1AGBjsQY04QlZJ1BVo7TlFB2ep3zXeFhPHnjOHomRDH7hVz2H6nybnzGmK6vpgJ2LrPmSmM6KUvIOoPG+cgKVpz2JZLjInnmlgkcra7jrhdX21QYxpgT7VwGdVUw2BIyYzojS8g6g16jIDL+tCaI9TS0VwKPXjuGdYWl/NubXxJq4yOMMa3YugBccU6NvDGm07GErDMIC4fMs09rpGVTM0b24oELBvHGmiJb69KYNhCRGSKyRUS2ichDzey/TUSKRWSd+3GHx756j+1Np/XpPFSd+ceyp0NEVKCjMcY0wxKyziJrsrOm5bGDHb7UfecN4qKRvfj3+ZtYlhda07oZ0x4iEg48AVwEDAdmicjwZg59TVXHuB/Pemyv9Nh+WTPndQ7Fm6Gs0JorjenELCHrLBqbETrQj6xRWJjwyDVnMLhnPPe+spadB491+JrGBKmJwDZVzVfVGuBVYGaAY/K+rQucZ+vQb0ynZQlZZ9FnLEREe6XZEiAuKoJnbplAmMAdL+RSXlXrlesaE2TSgUKP90XubU1dJSLrReR1EfFcEi7avc7uShG5vKWbBHw93ryF0HMUJPTx/72NMW1iCWBNwQEAABvkSURBVFlnEREJGWd2uGO/p74psTxx4zh2HDzGg6+to6HBOvkbcxreBfqp6mhgEfB/HvuyVHUCcAPwmIg0O8FXQNfjrSyFgpXWXGlMJ2cJWWeSlQP7NkBVmdcuOXlAGr+6dDgfbDrAo4u2eu26xgSJ3YBnjVeGe9tx/7+9Ow+TqrrzP/7+Vnf1Sjc00GwNNLuCsjeg3RoNUUZjVH7qT8UF9RnFRB10Mk4ivzFx1ImPY2YybiSKxkRccYlGjQ5NFDeisoOydkPYkR2aht77/P641VCNKFtX3Vo+r+epp6puLfd7tTn96XPOPdc5t8M5VxN6+jQwPOy1jaH71cCHwNBIFntcVn0ArgH6anV+kVimQBZLCovBNcL62S36tdeeVsi4kd14fGY57yze1KLfLRLn5gB9zaynmaUBVwLNzpY0s85hTy/CuzYvZpZnZumhx+3xLgO3NCpVH4uyGZCZB12L/K5ERL6DAlks6ToCAqktOmwJ3kXI773oVIoK87jz1UV8tbHleuBE4plzrh64DZiOF7Recc4tMbP7zKzprMmJZrbEzBYBE4HrQ9v7A3ND22cCD4auzxs7GhuhfAb0OcdbXkdEYlZELy4uxygty5vc30IT+5t9dWqA310znIsf/5QJU+fy1j+dQftWWo9IxDn3LvDuIdt+GfZ4EjDpMJ/7GzAw4gWeiE0LYN82DVeKxAH1kMWawmLYOB/qqlr8q/Nz0pkyvoid+2u55fn51NY3tvg+RCSGlJWCBaDPD/yuRESOQIEs1hSWQGMdbJgbka8/taA1D102mNlrdvLvby+JyD5EJEaUTfemQmS19bsSETkCBbJY020UYBEZtmxy0eAu/OTs3rz4xTqe+3xtxPYjIj7au8UbstRisCJxQYEs1mS2gU6ntvjE/kPdOeYkRp/cgXvfWsLnq3dEdF8i4oPyv3r3CmQicUGBLBYVlnhLX9TXRmwXKQHj4SuHUNgui1temM/6nfsjti8R8UHZdMjpAp1i+7wDEfEokMWiwmKor4LNiyK6m9yMIE9fN4L6hkZumjqX/bX1Ed2fiERJQx2smgl9zwUzv6sRkaOgQBaLuhd79xEetgTo2T6bx64axsote7nz1UU4p8sricS9dZ9DTYWGK0XiiAJZLGqVD+37RXRif7iz+uUz6fz+vPvl1zz2QXlU9ikiEVQ2HQJB6HW235WIyFFSIItVhcXeX7mNDVHZ3Y1n9uSSoQX8ZsZKSpd8HZV9ikiErCyFHiWQ3srvSkTkKCmQxarCEqjZA1uis1aYmfHAJQMZ3K0N/zxtISu+3huV/YpIC9u1Brav0Or8InFGgSxWFTbNI4vOsCVARjCFKdcOJzs9lZumzmXXvsid5SkiEVI2w7vvp0AmEk8UyGJV667QuntUJvaH65ibwRPXDufrPdXc9tJ86ht0eSWRuLJyOrTtBe16+12JiBwDBbJYVljs9ZBF+czHYd3zeOCSgcwq38Gv3l0W1X2LyAmo3Q9rPtFwpUgcUiCLZYXFsH87bC+L+q4vG96VfzyjJ3+YtYZX5q6P+v5F5Dis+QTqq6GflrsQiTcKZLGssMS7Xxe9eWThJp1/Mmf2bc/db3zFvLW7fKlBRI5BWSkEsw+2HSISNxTIYlm73pDdIaoT+8OlpgR4bNxQOrfJ4Obn5rF5T5UvdUiCqd0PK97zu4rE45y33EWvsyE13e9qROQYKZDFMrOD88h80iYrjafHF1Fd18CEqfOorovOumiSgOprYfZT8OgQeGkc7Frrd0WJZdty2LNOw5UicUqBLNYVlsCe9bB7nW8l9O2Yw8NXDOGrTXu46/XFurySHJvGBlg0DSaPgHfv9M4AvOE9yCv0u7LEUlbq3fc51986ROS4KJDFOh/WIzuccwZ05M4xJ/Hmwk1M+Xi1r7VInHDOG5p84gx4YwKk5cBVr3phrPB0v6tLPCtLoeNAaF3gdyUichwUyGJdhwGQ0Trq65Edzi1n9+aCQZ158H+XM3PFVr/LkVi25lP4/Rh46UrvrL9Lfw83f+wNp5n5XV3iqdoN6z7TcKVIHFMgi3WBAHT3dx5ZEzPj15cNon+nXCa+tIBV2yr9LklizaaF8Nwl8McLvKH2Hz0Mt86GgZd5P8sSGatngmuAvgpkIvFKLWQ8KCyGHeWwd4vflZCVlspT1xWRlhLgpmfnsqeqzu+SJBZsL4NXroMpZ8Gm+XDu/TBxARTdAClBv6tLfCtLITMPuo7wuxIROU4KZPHA5/XIDlXQJpPfXTOcdTv3c/vLC2ho1CT/pLVnA/z5Npg8yruG4vd+BrcvgpKJEMz0u7rk0NgI5TOgzzkQSPG7GhE5Tgpk8aDzIAhmxcSwZZORPdty78Wn8OGKbTw0fbnf5Ui07dsO//v/4NFhsHgajJzgBbHR/+bNeZTo2bwA9m3TcKVInEv1uwA5CilB6DYypgIZwNWjClm2uYInP1pN/065jB2qs7sSXnUFfDYZPnsc6vbD4Kvg7J9Dm+5+V5a8VpaCBbweMhGJWwpk8aKwBGY+APt3QlZbv6s54J4LT6FsSyU/f30xvfKzGdS1jd8lSSTUVcOcp+GT/4aqndD/Qhj9C8g/ye/KpGy6N3cshtoFETl2GrKMF4XFgIP1X/hdSTPBlAC/vXoY7VulM2HqPLburfa7JGlJDfUw71l4bBiU/ht0Hgw3fQBXPK8wFgsqt8KmBdBXi8GKxDsFsnhRMBxS0mJu2BKgXat0nhpfxJ6qOn783Dxq6nV5pbjX2Ahf/Ql+Owrengg5nWD8WzD+Te9nUWJD2Qzvvu8/+FuHiJywiAYyMzvPzFaYWbmZ3fUt77nczJaa2RIzezGS9cS1YKb3izAGAxnAgC65/Pflg5m/bje/ePMrXV4pXjkHZX/1lq947QYIpMIVL8CN70Ovs/yuTg5VNh1yOkOngX5XIiInKGJzyMwsBZgMnAtsAOaY2VvOuaVh7+kLTAJKnHO7zKxDpOpJCIXFMOsRqKmE9FZ+V/MNPxzYmYmj+/DoB+UM6JzL9SU9/S5JjsW6L+D9e72rQrTpDmOfgEGXaymFWNVQB6tmwiljdfUDkQQQyR6ykUC5c261c64WeBm4+JD33ARMds7tAnDO6Xo836WwGBrrYcMcvyv5Vnec048xAzpy/1+WMat8u9/lyNHYsgRevBKeGeMt8Hr+r+G2uTBknMJYLFv3OdRUaLhSJEFEMpAVAOvDnm8IbQvXD+hnZrPM7HMzOy+C9cS/bqO809tjdNgSIBAwfnPFEHrnZ3Pri/NZt2O/3yXJt9m5Gl6/CX5X4v1Mjf4F3L4QRk2A1HS/q4uaI02tMLPrzWybmS0M3W4Me+06MysL3a6LauFl0yEQhF5nR3W3IhIZfk/qTwX6AmcD44CnzOwb6yaY2QQzm2tmc7dt2xblEmNIeo53llsMBzKAVumpPDW+CIAbp86hsqbe54qkmb1fwzs/hcdHwLK3oeR2L4h9705Iy/a7uqgKm1pxPjAAGGdmAw7z1mnOuSGh29Ohz7YF7gFG4Y0I3GNmeVEq3ZvQ36MkJqcviMixi2Qg2wh0C3veNbQt3AbgLedcnXPu78BKvIDWjHNuinOuyDlXlJ+fH7GC40JhiTdkWV/jdyXfqbBdNpOvGsaqbfv46bSFNOrySv6r2gUz7oFHhsD8Z2HYdd71Js+9N5nXsDqaqRXf5h+AGc65naFpFzOA6PTy71oL25ZruFIkgUQykM0B+ppZTzNLA64E3jrkPW/i9Y5hZu3xhjBXR7Cm+FdYDA01sHG+35UcUUmf9tx9QX9Kl27h4ffL/C4nedXug4//Cx4e7J0U0v9CuG0O/Og3kNvZ7+r8djRTKwAuNbPFZvaamTX9oXm0n215ZaXevS6XJJIwInaWpXOu3sxuA6YDKcAzzrklZnYfMNc591botTFmthRoAP7VObcjUjUlhO6ne/drZ0Hh6f7WchSuL+7Bss0VPPp+Gf075XD+wKQPANFTXwvz/ggf/xr2bYV+58Pou6HTqX5XFm/eBl5yztWY2c3As8DoY/kCM5sATADo3r0FLjNVVgpte0H7Pif+XSISEyJ66STn3LvAu4ds+2XYYwf8NHSTo5HVFvL7x/w8siZmxv1jT6V8ayU/fWURhe2yGdAl1++yEltjAyx+BT58AHav84a5r3geuo/yu7JYdMSpFYf8kfg08FDYZ88+5LMfHm4nzrkpwBSAoqKiExu/r90Pf/8Yht9wQl8jIrHF70n9cjwKi71LKDXEx2T59NQUnrh2OK0zg9w0dS4799X6XVJicg6WveOdNfnmjyGjDVz9Olz/F4Wxb3fEqRVmFt6texGwLPS4qYc/LzSZf0xoW2St+RTqq3W5JJEEo0AWjwqLobYSvl7sdyVHrUNOBlPGD2d7ZQ23vDCPuoZGv0tKLKs/gqfPgWlXQ2MdXPYHmPAR9D1Hi4Z+B+dcPdA0tWIZ8ErT1Aozuyj0tomhK4ksAiYC14c+uxO4Hy/UzQHuC22LrLLpEMyGHmdEfFciEj0RHbKUCCks9u7X/g0KhvlbyzEY1LUN/3npIO6YtpD73l7K/WM1l+mEbZwH798Hqz+E3AK48FEYcjWk6J/20TqKqRWT8K4ocrjPPgM8E9ECm+8QVpZ6a48l0VpxIslArXY8yu0CeT1h3WdQfJvf1RyTsUMLWLa5gic/Xk3/zrlcNaoFJjgno20r4IP/gGVvQWZbGPMrGHEjBDP8rkwiadsK2LMOztS0W5FEo0AWrwpLYMW70NgIgfgaef7ZeSezYstefvnnr+jToRUjeybtGljHbvd6+PBBWPQiBLPgrLvg9FshQydKJIWy0BQ1LXchknDi6ze5HFRYDFU7YfsKvys5ZikB45Erh9K9XRY/eX4eG3dX+V1S7KvcBu/dBY8Ngy9fhVE/gdsXwfcnKYwlk5Wl0HEgtI7OcmciEj0KZPHqwDyyWf7WcZxaZwZ5anwRtQ2NTJg6l6raBr9Lik3Ve+CDX8Ejg2H2kzDoCpg4H857ALLb+12dRFP1Hm+ags6uFElICmTxKq8H5HSJm/XIDqd3fiseHTeUpZsr+NfXFuEtSycA1FXBrEe9IPbxQ94v4Vtnw8WPQ+uuflcnflj1AbgG6KfLJYkkIs0hi1dmXi/Z2lnemVdxurTB90/qwM/PO5kH31tO/8653Pr9JF95vKEOFjwPHz0EezdB79Hwg19Cl6F+VyZ+W1kKmXnQdYTflYhIBCiQxbPCYvjqNdj1d+8yKnHq5u/1YvnmCv6rdAUndczhnAEd/S4peuprYctX3vIVG+fDmk9gz3rvl+4lU6DnmX5XKLGgsRHKZ0DvH0Agxe9qRCQCFMjiWWGJd7/2b3EdyMyMBy8dxOrt+7hj2kLeuKWYvh1z/C6r5TkHO1d7wWvjPNg4FzYv9i4WD5DVHroWwfkPwUnnx22vp0TA5gWwb5uGK0USmAJZPMs/CbLaeYFs6DV+V3NCMoIpPHntcC58bBY3TZ3Ln289g9ZZQb/LOjH7doSCVyh8bZwHVbu811IzocsQGHkTFAz3bm26K4TJ4a0sBQz6nON3JSISIQpk8cwMup/uXdsujueRNencOpMnrx3GuClfcNtL8/nD9SNITYmT807qqrzervDwtWtN6EWDDv3h5AugoMgLXx36Q0qcB06JnrJSbxg7S2v2iSQqBbJ4V1gCy9+B/+wB7fpAu94H79v29u7T42f4b3hhW/5j7Kn87PXFPPjecu7+0QC/S/qmxkbYvrJ5+NqyBBpDF3vPLfAuaTX8Bi98dRkSV/8PJMZUboVN82H03X5XIiIRpEAW74aNBwt4AWFHOayZBYunNX9Pq06hoBYKa22b7nvG5PXwLh/RjaWbK3j607/Tv3Mulw73eZmHis3Nw9fGBVC713stLQcKhkLxxINDj7md/a1XEkvZDO++r+aPiSQyBbJ4l94KTvtx8221+70zL3eUw45VoVs5rHjPmxjcxALemlbt+jQPau16e/OZfDyb6+4L+rNyy14mvfElvfKzGdo9Lzo7rtkLmxaGha/5ULHRey2QCh1PgUGXHwxf7fvF3aWrJM6UlUJOZ+g00O9KRCSCFMgSUVqWFxw6nvLN16p2w85VsGN1KLCVe88XvQw1FQffFwh6PWjt+nhncDaFtna9vV8OEZ6vlpoSYPJVw7h48ixufm4eb//TGXTMbeELZzfUw9alYb1f82HbcnCN3ut5Pbw5egXDvbMfOw2EYGbL1iDyXRrqvAVhTxkb93NEReS7KZAlm8w2B3t3wjnn9Z419aYdCGurofz9g0szAASzoV2vsB61sLDWgpOO87LTeGp8EZf8dhYTnpvHtAmnkRE8zl4752D3urCzHud5PWH1oetoZuZ5E+77X+SFry7DILtdix2LyHFZ/4X3h5KGK0USngKZeMygVQfvVnh689caG6FiQ1hYW+X1qn29GJa97V3OpUlm3jeHP5tOMEhvdcxlndQph/+5YggTnpvHDx/5hIK8THIyUsnNCB64z80MNt+WGaS17aP1ri/J3LKAwKbQul9Nw7Up6dB5MAy/PtT7NRzyeqoHogXV1DdQWV1PZU09e6vr2VfjPW56XllTz41n9Iyfs2j9snK611vd6yy/KxGRCFMgkyMLBLw5ZW26Q+/vN3+tvtbreQof/txR7q04v/jl5u/N6XzwzM/wM0LzenznyQVjTunEQ5cN4i+LN1NRXcfmPdVUVNWxt7qeqroG0qijv61lSGAVgwOryLdyugS+PvD5VRSwInAqqzJPZkP2AHa36kt2Zia5dUFyvk4ldzfkZq4nJyPYLNTlZqSSkxEkLTU5QoNzjqo6L0jtralvFqgqa+qprK7znoe9duh7m7bVNjQecX/jRnSndVZy/Lc9bmWl0KNEZ+mKJAEFMjkxqWnQvo93O1Ttfm/I80BQC4W15X+B/dsPvs8C0Lpb86HPprDWuhsEUri8qBuXF3Xzeut2rj4w6b5xwzxsy5dYQ623y4x8duUNYnHu5WzIGsDqYD+212dQUV1HRVU9e6vrqNhTz94tO71QV1PPka5pnhEMhMJaaqg37mBYy80M9dIdeK2pp+7ga1lpKVgEe98aGl2zMFRZUxcWopqHqn2HCVTe617YajyK67unpwbIyUilVXoqrUL3XdpkhD0PHnw9tC0n7L3e8yAZQYWx77RrrTencdh1flciIlGgQCaRk5YFnU71boeq2nXwxIKdYfPW1s8+uKQEQEqaN5zYro8332vjPKje470WzCbQZSiM+rE376tgOGm5BXQ0oyMw6ChKbGx07Kutp6I6FNaaQluzxwdfq6iuY09VHRt27qeiup6Kqroj9galBIycjNSDQ6xhvXCHDrvmpKdS29D4LYGq7rC9UvtrG75z/02y01LCQlGQnPRU8lulH9h2aMjKCYWr8Ney01OTpsfQd2Wl3n3fMf7WISJRoUAm/sjM8+ZudT3MyQWVWw8JaqH7QBAGjD0Qvsg/+YSX5ggEjJxQjxYc3xmU1XUN7K32wtreUEgLf9z0WvjjdTv3h97nhapvrc8gJ+NgKMpOT6VNVhpd22Z5vU6HC1CHCVjZaamkBDRHLq6UzfDOcD5c77OIJBwFMoktZpDT0bv1KPG7mqOSEUwhI5hCfs7xLbLb0OioDAt0GcFAs2G9SA53Sgz7P0948zNFJCkokIn4LCVgtM4Kxv/F1KVlZbXVtStFkogmg4iIiIj4TIFMRERExGcKZCIiIiI+UyATERER8ZkCmYiIiIjPFMhEREREfKZAJiIiIuIzBTIRERERnymQiYiIiPhMgUxERETEZ+ac87uGY2Jm24C1x/CR9sD2CJUTK5LhGEHHmUiO9RgLnXP5kSommo6xDUuGnwVIjuNMhmMEHefhHFX7FXeB7FiZ2VznXJHfdURSMhwj6DgTSTIcY0tIlv9OyXCcyXCMoOM8ERqyFBEREfGZApmIiIiIz5IhkE3xu4AoSIZjBB1nIkmGY2wJyfLfKRmOMxmOEXScxy3h55CJiIiIxLpk6CETERERiWkJG8jM7DwzW2Fm5WZ2l9/1RIKZPWNmW83sK79riSQz62ZmM81sqZktMbPb/a6ppZlZhpnNNrNFoWO81++aIsnMUsxsgZm943ctsUptWGJIhvYLkqsNi1T7lZCBzMxSgMnA+cAAYJyZDfC3qoj4I3Ce30VEQT3wL865AcBpwK0J+P+zBhjtnBsMDAHOM7PTfK4pkm4HlvldRKxSG5ZQkqH9guRqwyLSfiVkIANGAuXOudXOuVrgZeBin2tqcc65j4GdftcRac65zc65+aHHe/H+IRT4W1XLcp7K0NNg6JaQEzzNrCtwAfC037XEMLVhCSIZ2i9InjYsku1XogayAmB92PMNJOA/gGRkZj2AocAX/lbS8kLd4AuBrcAM51zCHWPIw8DPgEa/C4lhasMSUCK3X5A0bVjE2q9EDWSSgMysFfA6cIdzrsLvelqac67BOTcE6AqMNLNT/a6ppZnZj4Ctzrl5ftciEk2J3n5B4rdhkW6/EjWQbQS6hT3vGtomccrMgniN2QvOuT/5XU8kOed2AzNJzLk1JcBFZrYGbxhutJk9729JMUltWAJJpvYLEroNi2j7laiBbA7Q18x6mlkacCXwls81yXEyMwN+Dyxzzv3G73oiwczyzaxN6HEmcC6w3N+qWp5zbpJzrqtzrgfev8sPnHPX+FxWLFIbliCSof2C5GjDIt1+JWQgc87VA7cB0/EmUL7inFvib1Utz8xeAj4DTjKzDWb2j37XFCElwLV4f40sDN1+6HdRLawzMNPMFuP9Mp7hnNOSEElKbVhCSYb2C9SGnTCt1C8iIiLis4TsIRMRERGJJwpkIiIiIj5TIBMRERHxmQKZiIiIiM8UyERERER8pkAmCcPMzjYznWYtInFJbVhyUyATERER8ZkCmUSdmV1jZrNDCyQ+GbogbaWZ/Y+ZLTGz980sP/TeIWb2uZktNrM3zCwvtL2Pmf3VzBaZ2Xwz6x36+lZm9pqZLTezF0KrZIuItBi1YRIJCmQSVWbWH7gCKAldhLYBuBrIBuY6504BPgLuCX1kKvBz59wg4Muw7S8Ak51zg4FiYHNo+1DgDmAA0AtvlWwRkRahNkwiJdXvAiTp/AAYDswJ/eGXCWwFGoFpofc8D/zJzFoDbZxzH4W2Pwu8amY5QIFz7g0A51w1QOj7ZjvnNoSeLwR6AJ9G/rBEJEmoDZOIUCCTaDPgWefcpGYbzX5xyPuO95peNWGPG9DPuIi0LLVhEhEaspRoex+4zMw6AJhZWzMrxPtZvCz0nquAT51ze4BdZnZmaPu1wEfOub3ABjMbG/qOdDPLiupRiEiyUhsmEaHkLVHlnFtqZncDpWYWAOqAW4F9wMjQa1vx5mgAXAc8EWqsVgM3hLZfCzxpZveFvuP/RvEwRCRJqQ2TSDHnjrdXVaTlmFmlc66V33WIiBwPtWFyojRkKSIiIuIz9ZCJiIiI+Ew9ZCIiIiI+UyATERER8ZkCmYiIiIjPFMhEREREfKZAJiIiIuIzBTIRERERn/1/0e5eQKNkE8QAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x360 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "def plot_curves(history, metrics):\n",
    "    nrows = 1\n",
    "    ncols = 2\n",
    "    fig = plt.figure(figsize=(10, 5))\n",
    "\n",
    "    for idx, key in enumerate(metrics):  \n",
    "        ax = fig.add_subplot(nrows, ncols, idx+1)\n",
    "        plt.plot(history.history[key])\n",
    "        plt.plot(history.history['val_{}'.format(key)])\n",
    "        plt.title('model {}'.format(key))\n",
    "        plt.ylabel(key)\n",
    "        plt.xlabel('epoch')\n",
    "        plt.legend(['train', 'validation'], loc='upper left');    \n",
    "    \n",
    "    \n",
    "\n",
    "plot_curves(history, ['loss', 'accuracy'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "3bdfbq20V6zu"
   },
   "source": [
    "You can see that accuracy is at 77% for both the training and validation data, while loss bottoms out at about .477 after four epochs.\n",
    "\n",
    "Key point: You will typically see best results with deep learning with much larger and more complex datasets. When working with a small dataset like this one, we recommend using a decision tree or random forest as a strong baseline. The goal of this tutorial is not to train an accurate model, but to demonstrate the mechanics of working with structured data, so you have code to use as a starting point when working with your own datasets in the future."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "SotnhVWuHQCw"
   },
   "source": [
    "## Next steps\n",
    "The best way to learn more about classifying structured data is to try it yourself. We suggest finding another dataset to work with, and training a model to classify it using code similar to the above. To improve accuracy, think carefully about which features to include in your model, and how they should be represented."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Copyright 2021 Google Inc.\n",
    "Licensed under the Apache License, Version 2.0 (the \"License\"); you may not use this file except in compliance with the License. You may obtain a copy of the License at\n",
    "http://www.apache.org/licenses/LICENSE-2.0\n",
    "Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License."
   ]
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "feature_columns.ipynb",
   "private_outputs": true,
   "provenance": [],
   "toc_visible": true
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
